Containers (meaning Docker) happened because CGroups and namespaces were arcane and required lots of specialized knowledge to create what most of us can intuitively understand as a "sandbox".

Cgroups and namespaces were added to Linux in an attempt to add security to a design (UNIX) which has a fundamentally poor approach to security (shared global namespace, users, etc.).

It's really not going all that well, and I hope something like SEL4 can replace Linux for cloud server workloads eventually. Most applications use almost none of the Linux kernel's features. We could have very secure, high performance web servers, which get capabilities to the network stack as initial arguments, and don't have access to anything more.

Drivers for virtual devices are simple, we don't need Linux's vast driver support for cloud VMs. We essentially need a virtual ethernet device driver for SEL4, a network stack that runs on SEL4, and a simple init process that loads the network stack with capabilities for the network device, and loads the application with a capability to the network stack. Make building an image for that as easy as compiling a binary, and you could eliminate maybe 10s of millions of lines of complexity from the deployment of most server applications. No Linux, no docker.

Because SEL4 is actually well designed, you can run a sub kernel as a process on SEL4 relatively easily. Tada, now you can get rid of K8s too.

My headcanon is that Docker exists because Python packaging and dependency management was so bad that dotCloud had no choice but to invent some porcelain on top of Linux containers, just to provide a pleasant experience for deploying Python apps.

I used cgroups, lxc, chroots, self-extracting executables. I built rugged, portable applications for UNICEF laptops and camps before docker was a thing.

And I think this whole point about "virtualization", "security", making the most use of hardware, reducing costs, and so on, while true, it's an "Enterprise pitch" targeted at heads of tech and security. Nice side effects, but I couldn't care less.

There are real, fundamental benefits to containers for a solo developer running a solo app on a solo server.

Why? My application needs 2 or 3 other folders to write or read files into, maybe 2 or 3 other runtime executables (jvm, node, convert, think of the dozens of OSS CLI tools, not compile-time libraries), maybe apt-get install or download a few other dependencies.

Now I, as a indie developer, can "mkdir" a few files from a shell script. But that "mkdir" will work the first time. It will fail the second time saying "directory already exists". I can "apt-get install" a few things, but upgrading and versioning is a different story altogether. It's a matter of time before I realize I need atleast some barebones ansible or state management. I can tell you many times how I've reinvented "smallish" ansible in shell scripts before docker.

Now if I'm in an enterprise, I need to communicate this entire State of my app – to the sysadmin teams. Forget security and virtualization and all that. I need to explain every single part of the state, versions of java and tomcat, the directories, and all those are moving targets.

Containers reduce state management. A LOT. I can always "mkdir". I can always "apt-get install". It's an ephemeral image. I don't need to write half-broken shell scripts or use ansible or create mini-shell-ansible.

If you use a Dockerfile with docker-compose, you've solved 95% of state management. The only 5% left is to docker-compose the right source.

Skip the enterprisey parts. A normal field engineer or solo developer, like me, who's deploying a service on the field, even on my raspberry pi, would still use containers. It boils down to one word: "State management" which most people completely underestimate as "scripting". Containers grant a large control on state management to me, the developer, and simplify it by making it ephemeral. That's a big thing.

My take: containers forced devepopers to declare various aspects of the application in a standardized, opinioated way:

- Persistant state? Must declare a volume. - IO with external services? Must declare the ports (and maybe addresses). - Configurable parameters? Must declare some env variables. - Trasitive dependecies? Must declare them, but using a mechanism of your choosing (e.g. via the package manager of your base image distro).

Separation of state (as in persistency) and application (as in binaries, assets) makes updates easy. Backups also.

Having most all IO visible and explicit simplifies operation and integration.

And a single, (too?!?) simple config mechanism increases reusability, by enabling e.g. lightweight tailoring of generic application service containers (such as mariadb).

Together this bunch of forced, yet leaky abstractions is just good enough to foster immense reuse & composability on to a plethora of applications, all while allowing to treat them almost entirely like blackboxes. IMHO that is why OCI containers became this big, compared to other virtualization and (application-) cuntainer technologies.

The article seems to assume that containers appeared to solve the software distribution problem and then somehow got repurposed into virtualization, isolation and management of production services. I think this view is very far from truth.

The virtualization/isolation aspect came first, the SWSoft Virtuozzo was doing that quite well in early 2000s. They even had [some] IO isolation which I think took around a decade to support elsewhere. Then gradually pieces of Virtuozzo/OpenVZ reached the mainline in a form of cgroups/LXC and the whole thing slowly brewed for a while until the Docker added the two missing pieces: the fast image rebuilds and the out-of-the-box user experience.

Docker of course was the revolution, but by then sufficiently advanced companies have been already using containers for isolation for a full decade.

Containers happened because nobody can be bothered to build an entire application into a single distributable executable anymore - heck even the tooling barely exists anymore. But instead of solving problems like dependency management and linking, today's engineers simply build piles of abstraction into the problem space until the thing you want to do more than anything (i.e. execute an application) becomes a single call.

Of course you now need to build and maintain those abstract towers, so more jobs for everybody!

> What people do with Docker is spin up a database or another service to develop or test against.

Yep. Being able to run

    docker run --rm --publish=127.0.0.1:27017:27017 'mongo:3.6.8'

    docker run --rm --publish=127.0.0.1:27017:27017 'mongo:5.0'

The author suggests that Docker doesnt help development and that devs just spin up databases, but I have to disagree with that and Im pretty sure i am not the only one.

All my projects (primarily web apps) are using docker compose which configures multiple containers (php/python/node runtime, nginx server, database, scheduler, etc) and run as a dev environment on my machine. The source code is mounted as a volume. This same compose file is then also used for the deployment to the production server (with minor changes that remove debug settings for example).

This approach has worked well for me as a solo dev creating web apps for my clients.

It has also enabled extreme flexibility in the stacks that I use, I can switch dev environments easily and quickly.

Fascinating documentary on Kubernetes for those who have 50 minutes. Gives more background to the "Container Wars". The filmmakers also have documentaries on the history of Python, Argo, etc.

Some highlights:

- How far behind Kubernetes was at the time of launch. Docker Swarm was significantly more simple to use, and Apache Mesos scheduler could already handle 10,000 nodes (and was being used by Netflix).

- RedHat's early contributions were key, despite having the semi-competing project of OpenShift.

- The decision to Open Source K8S came down to one meeting brief meeting at Google. Many of the senior engineers attended remotely from Seattle, not bothering to fly out because they thought their request to go OS was going to get shutdown.

- Brief part at the end where Kelsey Hightower talks about what he thinks might come after Kubernetes. He mentions, and I thought this was very interesting ... Serverless making a return. It really seemed like Serverless would be "the thing" in 2016-2017 but containers were too powerful. Maybe now with KNative or some future fusing of Container Orchestration + K8S?

[1] - https://youtu.be/BE77h7dmoQU

FreeBSD jails years ago based upon a user request.

>hosting provider's ... desire to establish a clean, clear-cut separation between their own services and those of their customers

https://en.wikipedia.org/wiki/FreeBSD_jail

My guess Linux started getting requests rom various orgs for a while, so in true Linux fashion, we got a a few different container type methods years later.

I still think Jails are the best of the bunch, but they can be a bit hard to setup. Once setup, Jails works great.

So here we are :)

Money.

Containers were around for a decade or more on FreeBSD and Solaris. They let you divvy up expensive big Unix iron.

Same as VMs were around on mainframes from the late 1960s and expensive Unix RISC servers from the late 1980s.

Linux didn't need it because it was cheap. So Linux replaced that older more expensive stuff, on cheap COTS hardware: x86.

Once everything was commoditised and cost-cut, suddenly, efficiency started to matter, so VMware thrived and was copied and VMs were everywhere.

Then the low usage and inefficiency of resource sharing of VMs made them look expensive, so they started to get displaced by the cheaper easier tech of containers, making "it works on my machine, so let's ship my machine" scale to production.

Containers happened because running an ad network and search engine means serving a lot of traffic for as little cost as possible, and part of keeping the cost down is bin packing workloads onto homogeneous hardware as efficiently as possible.

https://en.wikipedia.org/wiki/Cgroups

(arguably FreeBSD jails and various mainframe operating systems preceded Linux containers but not by that name)

Linux CGroups specifically were started at Google because their cluster management system Borg (or maybe it was still Babysitter at the time) needed a way to do resource tracking and admission control. Here's a comment by one of original devs: https://news.ycombinator.com/item?id=25017753

> I was always surprised someone didn't invent a tool for ftping to your container and updating the PHP

We thought of it, and were thankful that it was not obvious to our bosses, because lord forbid they would make it standard process and we would be right back where we started, with long lived images and filesystem changes, and hacks, and managing containers like pets.

> Application composition from open source components became the dominant way of constructing applications over the last decade.

I'm just as interested in why this ^ happened. I imagine it's pretty unique to software? I don't hear of car companies publishing component designs free for competitors to use, or pharmaceuticals freely waiving the IP in their patents or processes. Certainly not as "the dominant way of" doing business.

I wonder if LLM coding assistants had come about earlier, whether this would have been as prevalent. Companies might have been more inclined to create more of their own tooling from scratch since LLMs make it cheap (in theory). Individuals might have been less inclined to work on open source as hobbies because LLMs make it less personal. Companies might be less inclined to adopt open-source LLM-managed libraries because it's too chaotic.

Because dependencies on Unix are terrible for some languages that assume things are installed globally.

it happened because the story of dependencies (system & application) was terrible. the ability to run the app on different distribution/kernel/compiler/etc was hard. there were different solutions like vagrant, but they were heavy and the DX wasn't there

I loved the assertion that AI ate up all the budget and that K8s is now "boring" technology. That's fine because it was getting pretty annoying with all the clone competitors for practically everything that were popping up every month!

Do you use K8s? No! That's old! I use Thrumba! It's just a clone of K8s by some startup because people figured out that the easiest way to make gobs of money is/was to build platform products and then get people to use them.

I love this sentence about DevOps "Somehow it seems easier for people to relate to technology than culture, and the technology started working against the culture."

The article is just wrong. Before Docker, there was OpenVZ and Virtuozzo. They were used to provide cheaper "dedicatd machine" hosting back around 2005.

Then the technology from OpenVZ slowly made its way into the mainline Linux, in the form of cgroups and namespaces. LWN called it a "container puzzle", with tens of moving pieces. And it was largely finished by early 2010-s.

I built my own container system in 2012 that used cgroups to oversubscribe the RAM, with simple chroot-based file namespaces for isolation. We even used XFS projects (raise your hand if you know what this is!) for the disk quota management. I remember that I had to use systemtap to patch the kernel to be able to find out which process died as a result of the OOM killer, there were no standard ways to do that.

We sold it as a part of our biotech startup to Illumina. Then we sold it again to Amazon as a part of another startup :)

The genius of Docker was the layered overlayfs-based image building. This one simple innovation made it possible to build images in a constructive way, without having to waste half an hour for each minor change. I was floored with its simplicity and power when I first saw it.

"The compute we are wasting is at least 10x cheaper, but we have automation to waste it at scale now."

So much this. keep it simple, stupid (muah)

What blew my mind and convinced me to only use immutable distros is the immutability of it.

For instance I could create my own login screen for an web service without having to worry about the package manager overriding my code, because I inject it into the container, which is already updated.

I can also forcefully reroute much easier ports or network connections the way I want it.

Pretty simple. Modern development cycles for the libs that software depends on now happen significantly faster that OS/Distros update. This is future shock. The symptoms of this disease express as containers.

Do your part. Don't code on the bleeding edge. Keep things stable for users.

> There was one key innovation, which was Docker Hub

Vagrant has had that for VMs long before.

Dockerfiles and Docker Hub are directly inspired from Vagrantfiles and the Vagrant Box library

When you launch a container (either through docker you manually through namespaces) you are effectively representing yourself to the kernel as a separate thing. This allows you to construct a completely separate environment when interacting with the kernel where none of your concerns are going to leak out and nothing you don't care for is going to leak in.

When people say that static executables would solve the problem they are wrong, a static executable just means that you can eschew constructing a separate file-system inside your container - and you will probably need to populate some locations anyway.

Properly configured containers are actually supposed to be secure sandboxes, such that any violation is a kernel exploit. However the Linux kernel attack surface is very large so no one serious who offers multi-tenant hosting can afford to rely on containers for isolation. They have to assume that a container escape 0day can be sourced. It may be more accurate to say that a general kernel 0day can be sourced since the entire kernel surface area is open for anyone to poke. seccomp can mitigate the surface area but also narrow down the usefulness.

This and the comments may miss the forest for the trees.

Enterprise software vendors sold libraries and then "application servers", essentially promising infrastructure (typically tied to databases).

Enterprise software developers -- Google in particular -- got tired of depending on others' licensed infrastructure. This birthed Spring and Docker, splitting the market.

(Fun aside: when is a container a vm? When it runs via Apple containerization.)

Because Linux devs generally suck at making portable packages that are easy to install.

A really good post that captures the IT perspective of infrastructure change and modernization over my career. They also hit the nail on the head that one of today's problems is itself automation, where we've built tools that let us create sprawling estates with no accountability for cost other than an opaque monthly bill, though I'd also chip in that said automation is often restricted to whoever your chief cloud provider is (AWS, Azure, etc).

Where I find myself advocating today is very much a "rational check" on infrastructure, and curtailing accordingly. We have the tooling to ensure high availability, but does everything need to be HA? Do our SLAs for enterprise tooling really need five-nines of availability, or can we knock some applications down to a limited schedule? Does dev/test need to be live 24/7, or can we power it off when not in use? Why are we only focusing on availability and not scalability? The list goes on, but they're also not popular in enterprises with entrenched politics, which admittedly is where I find myself struggling against the current. If my social chops were better, I suspect I'd thrive in consultancy doing just that.

All that being said, I do like containers that are done right (properly documented, secure-by-default, ready for scaling), and I continue driving more applications towards containerization in the enterprise where possible. They're the right solution for ~60-80% of enterprise use cases, with the difficulty being getting vendors on board with the idea that their software won't have a dedicated VM or hardware anymore (which everyone fiercely resists, because container-based licensing can be a PITA to them). For the rest, VMs are more than fine, and we have a growing number of ways for both to exist peacefully in the same environment. As this area of technology matures (along with "backporting" from hyperscalers to private cloud again) further, I'm really looking forward to managing global estates in smaller teams for bigger firms - things that VMs, Containers, and Infrastructure-as-Code allow.

Most of the responses here are knee-jerk.

I liked the article. It's close to my adverntures with containers. I think the invention of docker is indeed mostly packaging: Dockerfile (functional) is pretty neat, docker hub (addressing a container) is awesome, and the ENTRYPOINT in Dockerfile is great, it distinguishes Docker from .deb.

But indeed, beyond Dockerfile things are bleak. Docker compose never rose to my expectations. To get serious things you need load blancer, storage, addressing, and these are beyond traditional containers scope.

Containers were invented because Linux is incapable of reliably running software programs. The global pool of shared libraries is an abject failure and containers are a heavy handed workaround.

how did solaris zones get left out of the story?

containers happened because the original execution isolation environment(the process) was considered a lost cause, Processes shared too much with each other so additional isolation primitives had to be added, but they had to be sort of tacked on to the side because more important than security or correctness is backwards compatibility. so now containers are considered a different thing than processes when really they are process with these additional isolation primitives enabled.

Containers happened because nobody knew what the hell they were doing and still have no clue what the hell they are doing. Software by the deranged for the deranged.

> was always surprised someone didn't invent a tool for ftping to your container and updating the PHP.

No FTP needed, you can just mount the application directory.

> Docker also made Go credible as a programming language,

Can someone explain why Docker was beneficial for Go?

For me the main reason to use containers is "one-line install any linux distro userspace". So much simpler than installing a dozen VirtualBox boxes to test $APP on various versions of ubuntu, debian, nixos, arch, fedora, suse, centos etc.

I have no idea how close I am, but Linux seems to have no way of having multiple versions of a dependency at once like Windows does. So I suspect thats the main reason.

Because containers were lighter than VM's, made reproducible and modularizable how to build the VM contents, and "solved" Unix/POSIX process isolation / library management design flaws.

You can laugh or not but its because they never finished gnu/hurd :D

Why does every discussion about containers always end up revolving around linux and cgroups? Didn’t we have Zones and even Jails way back?

in some situations, the cloud is a no-go and the lead time to install new hardware is quite lengthy. in those case vm's or containers allow for rapid change at the software/OS level while the meat space still moves like a glacier.

sounds minor, but it is a Big Deal for some

The tool that connects into your container and updates the files was called "Tilt".

Fwiw the actual video that he links to is well worth a watch.

Featuring one of the most Justin intros ever.

Why it happened is not nearly as important as what it unveiled: that versioned immutable systems are the most powerful system design concept in history. Most people have not yet grasped what an insanely powerful concept it is. At some point in the future, maybe 50-100 years from now, someone will look back and say "holy shit; deploying web apps are where this concept came from?" I hope in my lifetime that people get it and start applying it to other system designs/components.

Likely because Plan9's 'everything-is-a-filesystem' failed.

> Docker was easy to adopt as it did not change very much about how you used software.

What?! Docker images should seperate the program from the data. Something casual image maintainers fail to adhere to. I find docker extreme hard to use for persistent state that is not a remote DB.

It is even less secure than VMs (which seem to be like swiss chess if you try to break out). (Security-wise: rkt failed five years ago, BSD jails could do it better?)

Also, Docker was supposed to solve performance issues of VMs, debloating them. Instead, we got hypervisors or full VMs as hosts for Docker images, just to get the image as a de facto meta package manager to ship software.

So for me, it can be convenient, but I have no idea what I am doing when adding make file like statements to a yml to run full containers just to run e.g. a web server with some dependecies. But the workflow for that was novell to me, who previously fought with ruby and python module installations on host system.

I just wanted to have two instances of two versions of Postgres installed, and have their data directories documented in a YAML file, and know that they aren't gonna fuck up anything else on my system if someone gets RCE or something odd

I love boring.

I mean, containers do lend themselves to cargo culting by their very nature.

[dead]

Original sin.