The road to platform

Docker

The immutable image format that facilitates this new model comes from an open source project called Docker. A Docker container is an abstraction of an underlying host. It consists of a layered filesystem where each layer is immutable. A common pattern is to have an operating system layer, a middleware layer, and then an application layer. Each layer obfuscates the layer below it. If an application layer contains a file that exists in the middleware layer, then the application layer file is the only one seen when the image is started. The middleware layer file still exists, but it can't be seen or used.

This can cause a lot of leaky abstraction issues if you don't understand how Docker works. For example, if you copy a bunch of files in one layer and chmod them in the next layer, then all files will exist in the image twice. This can quickly add up and cause a lot of problems in the system.

Docker also uses Linux features like cgroups, which Google contributed about a decade ago, and namespaces. At their most basic, cgroups determine how many resources a process can consume, and namespaces determine who can consume them.

Additional abstractions have also been added into Docker over the years, including networks, volumes, secrets, and services. They also added labels to allow for assigning metadata. Metadata is very important in these modern, abstracted, distributed systems. We can no longer discuss machines based solely on their name, location, or address. We now describe and reference them based on multiple attributes like location, type, function, features, etc. This results in a more flexible abstraction.