Thank you for sharing that!

Yes, I still use Docker. Although my way of using it has changed over time, like many others I assume.

My new project, Dagger, although it doesn't directly compete with Docker, does implement its own container runtime and orchestrator (for CI pipelines), and we recently added compatibility with Dockerfiles and Docker Compose files... So increasingly I find myself using Dagger instead of Docker, in the context of my CI pipelines.

On the other hand, I use docker compose on almost every personal project - like a minecraft server for my kids, home automation, AI hacks, etc. These days I just let my AI agent set it up for me, though :)

> Any interesting memories from that day in 2013?

That day is forever engraved in my memory... Off the top of my head:

I signed up for a lightning talk expecting an audience of 10-20 people, for a confidential demo of our unfinished project that we had so far only showed in 1-1 demos. Imagine my surprise when I learned Pycon does lightning talks on the main stage!

The previous speaker didn't show, so I started early. In the recording you can hear me joke that perhaps I should get his time allocation too - the MC was not amused.

I went over time and had to speedtalk through the end.. The MC once again was not amused and practically shoved me off stage :-D

We had a small sponsor booth at the even, where our small marketing team was prepared to pitch our product (DotCloud) but was thoroughly confused when a horde of people showed up after the talk, to ask about "Docker" - which as far as they knew was the founder's side project.

I drove down to Santa Clara on my newly acquired motorcycle - my very first trip on the freeway. I remember telling myself, "today is the day to take all the risks!"

The Video in Question: https://www.youtube.com/watch?v=wW9CAH9nSLs

I definitely enjoy learning other people's opinions on things I didn't realize I had an opinion on until I learned there was something to opine about.

To me the arbitrarily complex docker networking makes perfect sense. Especially in a compose setup. I want my containers to be able to talk to each other, but I don't want them on my local network or even accessible (sometimes) from the local machine. The various internal networks offer isolation of containers from the rest of the network stack. If I have a rails app in a docker compose configuration with PG, Redis, Elasticsearch, etc, in a development environment, most of the time I don't want the service/accessory containers anywhere near anything other than the app's container. And if I want several compose stacks to work together, I just put them on the same named network. I apologize if I misunderstood your point.

Docker was basically a commercialisation of lxc and now we got nspawn going full circle.

No that’s all you. He just made it easier.

but heroku and docker serve different purpose no ???

I did make a bunch of money with Docker, something I am eternally grateful for.

I did not start a band, but I did build a nice house, help my family etc.

I still program everyday, working on my second startup, https://dagger.io (please forgive the plug). We have a very active discord if you want to come say hi.

I was inspired by Go's use of URI as a namespace for user packages, and short names for the standard library. I thought it was an elegant way to allow everyone to host anywhere, and dodge a lot of complicated namespacing issues.

One design principle that was very important to me, was portability and avoiding fragmentation. I wanted every image name to have the same meaning in every engine, so that when writing a Dockerfile, you could have confidence that it would actually behave the same on every docker installation. In this we disagreed with other platforms (like Red Hat) who sought to "embrace and extend" Docker by changing the meaning of short names. We always fought this to avoid fragmentation of the platform. This did not make sense from Red Hat's perspective, since they were the platform - who cared about fragmenting Docker? That led to some negativity which is probably what you're remembering.

At the same time early Docker users wanted the ability to host their own images without being forced to use our central registry. Using Go's system solved both problems at once.

Yes, I was around for that. It was one of my skunkworks projects, as I had lost formal authority over engineering and product by then, and mostly used my "founder aura" to convince small pockets of people within the company to build cool things with me.

However Moby was a poor choice for a skunkworks project, it involved mostly communications and project management, which requires consistency and scale. Overall it was poorly executed because I failed to build enough support internally ("Solomon's at it again, let him do his thing"). Here I am trying to explain it on Twitter after the keynote, using a picture of a hand-drawn diagram, because nobody in marketing or product was assigned to help. https://x.com/solomonstre/status/855918630915133440

I think the goal of Moby was fundamentally correct: we were trying to replicate the Red Hat model, with well-scoped upstream components (linux -> runc, containerd..) integrated into a commercial product (Docker -> RHEL), with a faster-moving community project in between (Fedora -> Moby).

This was a way for us to clarify that Docker was a product, with a community of users, and Moby was an open source project, with a community of contributors. This clarity was very much needed, because different communities were projecting very different expectations on the brand "Docker", and these expectations conflicted with each other.

In a way, Moby aspired to be what CNCF ended up becoming - the "big tent" where the broader community meets to hammer out the general shape of the container stack. It seems obvious now that a single company can't be the sole authority over such a vast scope. But in the first years of Docker, we really did have that level of influence, because we literally invented the market, and brought the community together in meetups and an ever-growing Dockercon. It's only when Google and Red Hat pushed Kubernetes and CNCF, that our influence over the stack started to wane. Maybe it was for the better, I don't know if we would have been up to the task as designers and engineers. But at least there would have been a consistent design, with a Linus Torvalds figure willing and able to say "no" to stupid and redundant features. For a while I played that role - but then I was replaced by SIGs and committees.

Oh gosh, the choice of language is such a context-specific design decision. The first versions of what became Docker were written in Python. The choice of Go was a mix of nerdy excitement ("it's like the best of Python and C!"), and adoption strategy: back then devops tools were segmented by language runtime: ruby, python or Java mostly. Shipping a compiled binary allowed us to be adopted by all tribes.

None of these things can be transposed today, so I guess I can't answer...

For what it's worth, my current project (dagger) is written in Go.

We acquired a small team of Cambridge-based computer science & systems engineering wizards, called Unikernel. We then worked together to apply systems engineering to improving and expanding Docker in various ways.

The most successful project that we worked on together was "project Piñata" which later became Docker for Mac. It turns out bundling a Linux VM in a native Mac app, and integrating it into the host filesystem, networking stack etc requires lots of magic..

Now, most of the Unikernel team were very active OCaml developers, and created MirageOS among other ocaml projects. While working on Piñata, they simply reached for the tools they were most familiar with.

Ha ha, thank you for your trusting us back then! Hopefully it was a good experience? Scaling reliably on AWS was not so simple back then...

Digital meet cute right here!

To give a sense of time: Docker launched in 2013, but it was one of many iterations of the same project, which I had been working on continuously since 2008. Over those 5 years, virtually nobody cared or understood what the hell I was working on. I lived in France and had no connection to the tech industry.

So, not only did I not anticipate this kind of success, I did not even have a reference frame for what success might even look like. All I knew was, I had an itch that I had to scratch, and I had nothing better to do :)

There are many. But there is an alternative universe where I remained CEO instead of transitioning to CTO, and continued pursuing my product vision obsessively instead of compromising on it because of lack of self-confidence and a desire to be loved. I think I would have avoided a lot of the mistakes that the company made - but on the other hand, I would have made a fresh set of mistakes instead, so who knows what the outcome would have been!

Not luck: pay attention to what the community needs, and be in a position to execute when you spot it.

I feel grateful for having seen from the inside one of those rare "hypergrowth" moments, it's a terrifying and exhausting experience but I would do it all over again without hesitation.

I also feel lucky that people are still using my work, that it endures without me, and that it generates (mostly) positive reactions. There was a time where Docker had a lot of haters, which was hard on me. But with time I've noticed that the negativity gets mostly washed away, and what remains is positive interactions like the ones in this whole thread. So now Docker is this cool experience I share with lots of people around the world, and when we meet we get to talk about this shared interest. I love it.

Why?

Python and Go are both memory-managed languages. That is perhaps the hardest part of programming (outside of getting the code to do what is intended).

Go and Rust are both compiled, but Rust requires the developer to figure out all of the memory management before compilation, which slows down development to a crawl.

Almost no company will rationally choose Rust over Go if they know how the development speed tradeoff looks like.

This obsession with Rust is very hard to understand

This is a strange question because you could learn Dockerfile syntax in a single night. There are only 18 directives, and the average Dockerfile uses like 6 of them. 90%+ of Dockerfiles is just "run this shell command" and "copy this file into the docker image". The only time it flirts with complexity is multi-stage Dockerfiles, but 90%+ of Dockerfiles are a single stage.

https://docs.docker.com/reference/dockerfile/

I think the key thing for a university student is to get the big ideas down: what is a container exactly, how does it work, how does it differ from a VM, what assumptions does it make about what you're running (and how do you run something that doesn't fit that pattern), why do images exist in layers and how are layers implemented, what are the performance benefits/tradeoffs, etc. etc.

When I was learning Docker I can't tell you how many people would describe it to me by analogy to a standardized shipping container, and start going off about shipping container dimensions and supersized cargo ships and trucks and railcars... and finish by smiling beatifically, like they had just explained the whole thing in a masterstroke. Let's... uh... not do that. :-D

In a problem set, the student might learn a bit about dockerfiles or docker-compose.yml... but only, I think, so they can try building and running a container of their own!

I'm gonna be real with you. No one needs to know docker syntax. Most engineers just copy and paste chunks of Dockerfiles around anyways.

Docker syntax is one of those things that just immediately falls out of my head when I close the editor. I don't care about it at all.

Even better: there are many, many tools that build OCI images without using the Docker syntax. I haven't hand-written a Dockerfile in at least six years.

LLMs have not loved the needle here in my sphere, it was already bouncing off of the zero on the gauge.

edit: this isn't to say the syntax is good or bad (though there are several questionable parts of it) but extremely simple, and concise enough that you can just look up literally any command with great ease. It's surely a large part of its success, for better or for worse.

If you're going to fully delegate something so trivial as Dockerfiles to slopmachines, why go to university at all? You don't need to learn anything if you delegate everything to the slopmachine.

I assume you mean podman?

The first two are just kind of the nature of containers on linux, and applies to any container technology.

Podman does allow you to run it as a non-root user (with some limitations), that was in fact one of the biggest motivations for making podman. This wasn't possible at all in the early days of docker, it wasn't until linux introduced user namespaces, where the root user inside the namespace mapped to a non-root user id outside the namespace that non-root conatiners were possible.

And podman, unfortunately IMO, is also written in go.

I switched to Podman because it’s daemon less (no resources are used unless a container is running) and it doesn’t need to run as root.

Otherwise it’s basically the same and you can continue using your Dockerfiles and Docker Compose files.

Unless you’re doing something really fancy it’s basically a drop-in replacement.

I think you meant "Podman", but yes, Postman almost entirely sidesteps docker.

:D

I think the only thing in the "weird things" list that don't necessarily apply to another container system on linux would be the go-specific issues.

I’d argue in 2013, the client / server model is what made docker a success. It allowed for a seamless, and mostly identical, experience for docker on Mac, Windows and Linux for one. It met developers where they were, daily-ing a Mac or a Windows and deploying on Linux headless servers. It had benefit on its native Linux system too that took all the alternative solutions another 8 or 9 years for the entire Linux ecosystem to develop enough solutions around containers to make a daemon unnecessary while still offering docker DX.

That's actually a design decision made to work around the limitation of having to run as root. Since the container had to be started as a privileged user, if you want non-privileged users to be able to start contianers you either need a setuid binary or a client/server interface like docker has. There are pros and cons to each of those options.

Now it's possible to create containers without elevated privileges (assuming user namespaces are allowed), but that wasn't possible until long after docker had established the client/server model.

lxc never had a daemon but Canonical added one (lxd in 2015) and lxc forked it to incus to be community managed.

rkt (2013) was CoreOS's alternative to Docker and was daemon-less and failed.

Podman came along relatively late (2018) and benefited from a lot of Linux kernel and systemd improvements since Docker was designed.

So while arguably the client/server model is outdated for container management tools, I'm not sure it's weird. This is my workstation for example:

  schmichael@fedora:~$ ps -e -o comm= | rg 'd$' | wc -l
  67

Yeah, the daemon is the real problem with Docker.

This is why I prefer Podman.

Maybe. That's why i start thinking i am getting old and less curious :(

These days they may as well have the mental model of Docker when they start using a VM :)

That’s an absurd statement. Optimizing every language around what a runc-like program would need is absurd.

Where does this advice come from? Most web servers would use a lot more resources if they didn't use userland threading, wouldn't they?

Off: apparently both Firefox and Chromium gives F1-F8 to websites to use if they ask nicely.

Easy: far few footguns.

Performance is basically irrelevant because you're not really interacting with the running application. You're running as root, so security-critical. You're talking to the internet for things like fetching images. You need rather complicated business logic to deal with all the possible options, protocols, and file formats.

Sure, you could do it in C, but dealing with dependencies is a major PITA and you're permanently one tiny mistake away from getting pwned, while not needing any of C's strengths. Go is safe, easy to write, and ships as a single batteries-included binary: what's not to like?

That isn't really relevant for the tool starting the container. The biggest problem with go here is that it can't natively perform the operations necessary to set up a container, due to the nature of the runtime, so it has to resort to complicated workarounds like a shim in c, and starting spawning from /proc/self/exe instead of forking/cloning.

it changed and its better now. rootless dockerd wants newuidmap/newgidmap + subuid range before it start, and cgroups only work with cgroup v2 + systemd. also, on a host that already has those, its fine on one without systemd, and podman is still less friction, mostly because there is no daemon to babysit

Give them a break, machines with multiple CPU cores have only been around for 40ish years!