I have captured some of my learnings on Docker compose files and how they differ between versions. Docker compose is a tool used for defining and running multi-container Docker applications. I have used the famous multi-container voting application to illustrate the differences with compose versions.
Following are some questions that I have to tried to answer in this blog:
- What is the difference between Compose versions 1, 2 and 3?
- What is the difference between compose, stack and dab formats?
- What are different ways to run compose files with different compose versions?
- How does “docker stack deploy” really work?
Compose versions:
Following table captures the main differences between Compose versions:
| Feature | Compose v1 | Compose v2 | Compose v3 |
| Docker engine version | 1.9.1+ | 1.10.0+ | 1.13.0+ |
| Compose version | Compose up to 1.6.x | Compose 1.6.0+ | Compose 1.10.0+ |
| Connect containers | Bridge network, need links | Overlay supported, networks can be used to connect containers | Overlay supported, networks can be used to connect containers |
| Volume | No named volume | Named volume supported | Named volume supported |
| Swarm capability | Older swarm | Older swarm | Swarm mode |
| Deploy section | No deploy section | No deploy section | Deploy section with replica present |
There could be many reasons why someone would want to use older compose versions. It could either be because they are still using older Docker versions or they already have compose files that are currently in use. I have captured the famous votingapp in all 3 formats to capture the differences:
votingapp in compose v1:
Following is the votingapp in compose v1 format. This establishes container connectivity using links and can be deployed in standalone node or in cluster using old swarm mode.
vote:
image: docker/example-voting-app-vote:latest
environment:
- "constraint:node==swarm-agent-753F9D8C000000"
ports:
- "5000:80"
links:
- redis
redis:
image: redis:alpine
environment:
- "constraint:node==swarm-agent-753F9D8C000000"
ports: ["6379"]
worker:
image: docker/example-voting-app-worker:latest
environment:
- "constraint:node==swarm-agent-753F9D8C000000"
links:
- redis
- db
db:
image: postgres:9.4
ports: ["5432"]
environment:
- "constraint:node==swarm-agent-753F9D8C000000"
result:
image: tmadams333/example-voting-app-result:latest
ports:
- "5001:80"
environment:
- "constraint:node==swarm-agent-753F9D8C000000"
links:
- db
Following command can be used to deploy:
docker-compose up -d
votingapp in compose v2:
Following is the votingapp in compose v2 format. This establishes container connectivity using networks and can be deployed in standalone node or in cluster using old swarm mode. Containers connected through networks can be deployed in separate nodes and they can talk through overlay network here.
version: "2"
services:
vote:
image: docker/example-voting-app-vote:latest
labels:
- "com.example.description=Vote"
ports:
- "5000:80"
networks:
- front-tier
- back-tier
redis:
image: redis:alpine
ports: ["6379"]
networks:
- back-tier
worker:
image: docker/example-voting-app-worker:latest
networks:
- back-tier
db:
image: postgres:9.4
ports: ["5432"]
labels:
- "com.example.description=Postgres Database"
networks:
- back-tier
result:
image: tmadams333/example-voting-app-result:latest
ports:
- "5001:80"
networks:
- front-tier
- back-tier
networks:
front-tier:
back-tier:
Following command can be used to deploy:
docker-compose up -d
votingapp in compose v3:
Following is the votingapp in compose v3 format. This establishes container connectivity using networks and can be deployed in standalone node or in new swarm mode. This compose format has the deploy section that controls deployment parameters like number of replicas, constraints etc.
version: "3"
services:
redis:
image: redis:alpine
ports:
- "6379"
networks:
- frontend
deploy:
replicas: 2
update_config:
parallelism: 2
delay: 10s
restart_policy:
condition: on-failure
db:
image: postgres:9.4
volumes:
- db-data:/var/lib/postgresql/data
networks:
- backend
deploy:
placement:
constraints: [node.role == manager]
vote:
image: dockersamples/examplevotingapp_vote:before
ports:
- 5000:80
networks:
- frontend
depends_on:
- redis
deploy:
replicas: 2
update_config:
parallelism: 2
restart_policy:
condition: on-failure
result:
image: dockersamples/examplevotingapp_result:before
ports:
- 5001:80
networks:
- backend
depends_on:
- db
deploy:
replicas: 1
update_config:
parallelism: 2
delay: 10s
restart_policy:
condition: on-failure
worker:
image: dockersamples/examplevotingapp_worker
networks:
- frontend
- backend
deploy:
mode: replicated
replicas: 1
labels: [APP=VOTING]
restart_policy:
condition: on-failure
delay: 10s
max_attempts: 3
window: 120s
placement:
constraints: [node.role == manager]
networks:
frontend:
backend:
volumes:
db-data:
There are 2 ways to deploy an application in compose v3 format. The first is the traditional option using docker-compose.
docker-compose up -d
The above option ignores the parameters under deploy section.
The second preferred option is to use “docker stack” approach as shown below. With this, the Docker services gets directly deployed in the Swarm mode cluster.
docker stack deploy --compose-file vote
In “docker stack deploy”, compose file parsing is done by Docker engine itself compared to docker-compose binary handling compose file parsing. There are few options still missing in “docker stack deploy”. 1 of them is environment variables which has been addressed recently and should be available in 17.04 Docker version. Another is related to “extends” feature and is being discussed here. The general goal is get Docker engine have all features of docker-compose so that “docker stack deploy” would become the universal way to deploy compose files.
Difference between compose, stack and dab file formats:
Stack file is same as compose file with additional deploy section. Stack file is part of Compose v3 format. If we are using new Swarm mode, it is better to move to stack file format which is the same as compose v3.
Distributed application bundle(Dab) was introduced in Docker 1.12 as an experimental feature. dab file specifies container image with digest rather than defining container image with tags. dab file is auto-generated from docker-compose file. Many options of compose is not supported in dab. “docker stack” introduced in Docker 1.13 has the capabilities of compose and dab combined. My understanding is “docker stack” will make dab format obsolete. I checked with few folks in Docker team and this seems to be the case though it is not fully confirmed.
To generate dab from compose file, we can use the following command:
docker-compose bundle -o
To deploy the bundle, we can use the docker stack command:
docker stack deploy --bundle-file
Following is the votingapp in dab format:
{
"Services": {
"db": {
"Image": "postgres@sha256:c681c72ff62b637b9ad3806a946f14450f74ee0888ca70ec1472a4e0552054c2",
"Networks": [
"backend"
]
},
"redis": {
"Image": "redis@sha256:95f0c9434f37db0a4f9563379184ff4895685d2381e2d81bd10dfcbcadfc095f",
"Networks": [
"frontend"
],
"Ports": [
{
"Port": 6379,
"Protocol": "tcp"
}
]
},
"result": {
"Image": "dockersamples/examplevotingapp_result@sha256:83b568996e930c292a6ae5187fda84dd6568a19d97cdb933720be15c757b7463",
"Networks": [
"backend"
],
"Ports": [
{
"Port": 80,
"Protocol": "tcp"
}
]
},
"visualizer": {
"Image": "dockersamples/visualizer@sha256:f924ad66c8e94b10baeaf7bdb9cd491ef4e982a1d048a56a17e02bf5945401e5",
"Networks": [
"default"
],
"Ports": [
{
"Port": 8080,
"Protocol": "tcp"
}
]
},
"vote": {
"Image": "dockersamples/examplevotingapp_vote@sha256:8e64b18b2c87de902f2b72321c89b4af4e2b942d76d0b772532ff27ec4c6ebf6",
"Networks": [
"frontend"
],
"Ports": [
{
"Port": 80,
"Protocol": "tcp"
}
]
},
"worker": {
"Image": "dockersamples/examplevotingapp_worker@sha256:3e4dd0f59c15f432280a2c0679c4fc5a2ee5a797023c8ef0d3baf7b1385e9fed",
"Networks": [
"frontend",
"backend"
]
}
},
"Version": "0.1"
}
How does “docker stack deploy” really work?
I have prepared the flowchart below based on referring the notes from this issue. The flowchart below shows how the image specified in the compose file resolves to what gets deployed finally when “docker stack deploy” is used.
“docker stack deploy” is idempotent. If the stack is already deployed, “docker stack deploy” will restart only those services which has the digest or tag that is updated. In the above flowchart, the restart applies to the 2 cases where the latest image is pulled from registry.