osteel's blog Web development resources

How to use Docker for local web development: an update

Docker logo

A year or so ago I published a post titled From Vagrant to Docker: How to use Docker for local web development, which got quite some traction at the time and continues to get comments on a regular basis, even today.

Thing is, Docker (and its ecosystem) is a fast-paced project that gets updated very often, and things changed quite a bit since that article was first published.

Today's objective is to bring the concepts it exposed up to speed, and to provide an updated companion repository, which I will then use as a starting point for further Docker tutorials.

If you are new to Docker, I invite you to read the previous post nevertheless, as it will show you how to use Docker, and most of the knowledge it contains remains valid anyway.

Table of contents

Docker Toolbox or native app?

Whether you are coming from From Vagrant to Docker: How to use Docker for local web development or not, you might be using Docker Toolbox and Docker Machine as part of your current developing environment.

A few months after I published that article, Docker released native applications for both Mac and Windows, essentially abstracting away the management of the Virtual Machine and using the virtualization toolkit provided with the operating system.
If you have no idea what I am talking about or if you do and that sounds appealing, I would recommend to install the native app for your OS instead of Docker Toolbox (even though they can coexist).

But before you do so, make sure you read the What to know before you install section of each guide (Mac, PC), especially to make sure your system meets the requirements.

If it doesn't, then stick to Docker Toolbox.

The installation guides are pretty straightforward, so I'll leave you to it:

From Docker 1.9.x to 1.13.x

Docker has known a lot of improvements with the recent releases, among which enhanced security and networking, new versions for Docker Compose and its file format, and some significant progress with container orchestration as well as its swarm mode.

The focus has also been put on bringing together an industry standard, which has materialised in the Open Container Initiative in June 2015, providing specifications and promoting interoperability between the different actors of the container industry.

OCI logo

runC is notably born out of it, following the Runtime Specification (essentially describing how containers should be spawned and run), and Docker also focused on breaking down its engine into different components (such as containerd), allowing for more control and flexibility (you can read more about it in this Medium post).

To be honest though, the above has little impact on what we are trying to achieve here, and our main concern is essentially the updates brought to Compose and its file format.

Migrating our project

As aforementioned, Docker Compose and its file format were also updated, from version 1.5.x to 1.9.0 for the former, and from 1.0 to 2.1 for the latter.

These releases upgraded networks and volumes to first-class citizens among other things, placing them at the same level as services in the docker-compose.yml file.
The Docker documentation provides a convenient guide which is a good starting point to upgrade our file, but there is also a few extra steps to take.

Current state

Let's start with a quick reminder of what is currently contained (see what I did there?) in our project:

  • a container for Nginx
  • a container for PHP-FPM
  • a container for MySQL
  • a container for phpMyAdmin
  • a data-only container to make MySQL data persistent
  • a data-only container for the application code

Step 1 - add the version and the services key

We're starting off with the docker-compose.yml file as we left it at the end of the previous tutorial (you can clone its repository and go from there):

nginx:
    build: ./nginx/
    ports:
        - 80:80
    links:
        - php
    volumes_from:
        - app

php:
    build: ./php/
    expose:
        - 9000
    links:
        - mysql
    volumes_from:
        - app

app:
    image: php:7.0-fpm
    volumes:
        - ./www/html:/var/www/html
    command: "true"

mysql:
    image: mysql:latest
    volumes_from:
        - data
    environment:
        MYSQL_ROOT_PASSWORD: secret
        MYSQL_DATABASE: project
        MYSQL_USER: project
        MYSQL_PASSWORD: project

data:
    image: mysql:latest
    volumes:
        - /var/lib/mysql
    command: "true"

phpmyadmin:
    image: phpmyadmin/phpmyadmin
    ports:
        - 8080:80
    links:
        - mysql
    environment:
        PMA_HOST: mysql

The only thing we need to do for a start is to specify the format version at the top, and to place the rest of the file under a new services key:

version: "2.1"

services:
    nginx:
        build: ./nginx/
        ports:
            - 80:80
        links:
            - php
        volumes_from:
            - app

    php:
        build: ./php/
        expose:
            - 9000
        links:
            - mysql
        volumes_from:
            - app

    app:
        image: php:7.0-fpm
        volumes:
            - ./www/html:/var/www/html
        command: "true"

    mysql:
        image: mysql:latest
        volumes_from:
            - data
        environment:
            MYSQL_ROOT_PASSWORD: secret
            MYSQL_DATABASE: project
            MYSQL_USER: project
            MYSQL_PASSWORD: project

    data:
        image: mysql:latest
        volumes:
            - /var/lib/mysql
        command: "true"

    phpmyadmin:
        image: phpmyadmin/phpmyadmin
        ports:
            - 8080:80
        links:
            - mysql
        environment:
            PMA_HOST: mysql

Even though we won't need the new 2.1 features for now, that's the latest version so let's specify this one.

Open the project's folder in a new terminal window, and run:

$ docker-compose up -d

You should observe something similar to this:

Step 1

We can observe that the first line is mentioning a network being created - we'll get to that in a minute.

Now access http://localhost in your browser (if using Docker for Mac or Windows), and this is what you should see:

Step 1 result

Now stop and remove the containers and their volumes running the following command:

$ docker-compose down -v

This is a shortcut command that was introduced with Compose 1.6.0 (combining docker-compose stop and docker-composer rm -v).

Starting from version 1.6.0, Docker Compose automatically creates a network which is accessible by each container of each defined service, and on which they are all discoverable by default.

You guessed it, that's what the line describing the creation of a network we noticed in the previous step is about.

There is therefore no need for explicitly declaring links anymore (unless you need extra aliases), so you can remove them all.

Step 3 - depends_on

Also introduced with Docker Compose 1.6.0, this option allows to express dependency between services, making Docker Compose start the different services in dependency order.
Note that it won't wait for a service to be ready, but only for it to be started.

This is what the content of your file should now look like, with the depends_on options and without the links ones:

version: "2.1"

services:
    nginx:
        build: ./nginx/
        ports:
            - 80:80
        volumes_from:
            - app
        depends_on:
            - php

    php:
        build: ./php/
        expose:
            - 9000
        volumes_from:
            - app
        depends_on:
            - mysql

    app:
        image: php:7.0-fpm
        volumes:
            - ./www/html:/var/www/html
        command: "true"

    mysql:
        image: mysql:latest
        volumes_from:
            - data
        environment:
            MYSQL_ROOT_PASSWORD: secret
            MYSQL_DATABASE: project
            MYSQL_USER: project
            MYSQL_PASSWORD: project

    data:
        image: mysql:latest
        volumes:
            - /var/lib/mysql
        command: "true"

    phpmyadmin:
        image: phpmyadmin/phpmyadmin
        ports:
            - 8080:80
        depends_on:
            - mysql
        environment:
            PMA_HOST: mysql

When running docker-compose up -d again, we can see how this is affecting the order in which services are being started:

Step 3 before ^ before

Step 3 after ^ after

Step 4 - named volumes

We are now broaching one of the main additions of Compose 1.6.0.

As explained earlier, volumes and networks are now first-class citizens, and volumes can be declared at the same level as services.

In the case of MySQL, this means that we don't need the data-only container data anymore, and we are going to make it a named volume instead:

version: "2.1"

services:
    nginx:
        build: ./nginx/
        ports:
            - 80:80
        volumes_from:
            - app
        depends_on:
            - php

    php:
        build: ./php/
        expose:
            - 9000
        volumes_from:
            - app
        depends_on:
            - mysql

    app:
        image: php:7.0-fpm
        volumes:
            - ./www/html:/var/www/html
        command: "true"

    mysql:
        image: mysql:latest
        volumes:
            - data:/var/lib/mysql
        environment:
            MYSQL_ROOT_PASSWORD: secret
            MYSQL_DATABASE: project
            MYSQL_USER: project
            MYSQL_PASSWORD: project

    phpmyadmin:
        image: phpmyadmin/phpmyadmin
        ports:
            - 8080:80
        depends_on:
            - mysql
        environment:
            PMA_HOST: mysql

volumes:
    data:

We removed the data block under services, and added a volumes block at the end of the file. We simply declared a new data volume in there, with no other options (it uses the local driver by default). Then, we replaced the volume_from option from the mysql block with a volumes key, pointing /var/lib/mysql to the data volume.

Here, we are essentially asking Docker to copy whatever is inside /var/lib/mysql in the mysql container to the data volume when it is first initialised. Any update to that directory is now persisted, since the volume won't be destroyed along with the container (unless the -v option is used), and the volume and its data will be picked up next time a volume with the same name is invoked by a service.

Running docker-compose up -d again, we can see the volume being created on the second line:

Step 4

Good. But how about the app data-only container then?

Well, that's where it gets a bit tricky: for some reason, syncing a local directory with a named volume is not supported out of the box.

I am not the first one to wonder about it, and someone actually created a plugin for that. But, looking at the installation guide, I honestly wonder if it is worth the hassle (when you are using Docker for Mac like me, it basically requires the plugin to run in its own container while maintaining its state in a JSON file).

Somewhere in the issue's comments, someone suggests using the driver's specific options: this is not a viable solution either, since they are not cross-platform compatible (e.g. the local driver on Windows doesn't support any options, hence mounting a local directory wouldn't work).

We could still use a data-only container, but this is not a recommended practice anymore (the documentation states otherwise but is currently out of date).
Moreover, and as the local-persist plugin's author rightly points out in the documentation, using a data-only container implies mounting the local directory into the same path in all the containers that use it.

There is simply currently no easy way to sync a local folder with a named volume, so I decided to settle for basic bind-mounting for now, until a better solution comes up (if you know one, feel free to share it in the comments):

version: "2.1"

services:
    nginx:
        build: ./nginx/
        ports:
            - 80:80
        volumes:
            - ./www/html:/var/www/html:ro
        depends_on:
            - php

    php:
        build: ./php/
        expose:
            - 9000
        volumes:
            - ./www/html:/var/www/html
        depends_on:
            - mysql

    mysql:
        image: mysql:latest
        volumes:
            - data:/var/lib/mysql
        environment:
            MYSQL_ROOT_PASSWORD: secret
            MYSQL_DATABASE: project
            MYSQL_USER: project
            MYSQL_PASSWORD: project

    phpmyadmin:
        image: phpmyadmin/phpmyadmin
        ports:
            - 8080:80
        depends_on:
            - mysql
        environment:
            PMA_HOST: mysql

volumes:
    data:

This leads to some configuration repetition, but as we'll see later there is a way to somewhat mitigate this.

In the above code, you might have noticed the presence of the :ro option at the end of the volumes declaration of the nginx service: since this service doesn't need the write permission for this directory, this option gives it a read-only access instead.

Step 5 - networks

We already saw that from Compose 1.6.0, a default network to which all services have access is created. We could leave it like that, or we can choose to fine-tune that a bit and define a private networks for the relevant services instead.

Let's declare two of those - database and server, at the end of docker-compose.yml, right after the volumes section:

volumes:
    data:

networks:
    database:
    server:

The former will be used for services that need access to the database, and the latter for services dealing with HTTP requests.

Now change the definition of the services like so:

services:
    nginx:
        build: ./nginx/
        ports:
            - 80:80
        volumes:
            - ./www/html:/var/www/html:ro
        networks:
            - server
        depends_on:
            - php

    php:
        build: ./php/
        expose:
            - 9000
        volumes:
            - ./www/html:/var/www/html
        networks:
            - database
            - server
        depends_on:
            - mysql

    mysql:
        image: mysql:latest
        volumes:
            - data:/var/lib/mysql
        networks:
            - database
        environment:
            MYSQL_ROOT_PASSWORD: secret
            MYSQL_DATABASE: project
            MYSQL_USER: project
            MYSQL_PASSWORD: project

    phpmyadmin:
        image: phpmyadmin/phpmyadmin
        ports:
            - 8080:80
        networks:
            - database
        depends_on:
            - mysql
        environment:
            PMA_HOST: mysql

The mysql, php and phpmyadmin services are on the database network, and the nginx and php services are on the server one (meaning for instance that the nginx and mysql services are isolated from each other).

Now run docker-compose up -d again. You should get:

Step 5

We can observe that the default network is no longer created (another way to make sure of this is to run the ad hoc command docker network ls).

Then again, this step is rather optional, especially for a local development environment, but it shows how to add more security and isolation between the different services.


Note: if you are wondering why phpmyadmin doesn't need access to the server network, the reason is because the phpMyAdmin image we use is already shipping with its own versions of PHP and nginx.


Step 6 - .env file

Starting from version 1.7.0, Docker Compose will look for a .env file in the directory where it's run from and, if it finds one, will read the environment variables it contains to replace some placeholders with the corresponding values.

Let's create such a file at the same level as docker-compose.yml, with this content:

PROJECT_ROOT=./www/html

DB_ROOT_PASSWORD=secret
DB_NAME=project
DB_USERNAME=project
DB_PASSWORD=project

And update the definition of the services, one more time:

services:
    nginx:
        build: ./nginx/
        ports:
            - 80:80
        volumes:
            - "${PROJECT_ROOT}:/var/www/html:ro"
        networks:
            - server
        depends_on:
            - php

    php:
        build: ./php/
        expose:
            - 9000
        volumes:
            - "${PROJECT_ROOT}:/var/www/html"
        networks:
            - database
            - server
        depends_on:
            - mysql

    mysql:
        image: mysql:latest
        volumes:
            - data:/var/lib/mysql
        networks:
            - database
        environment:
            MYSQL_ROOT_PASSWORD: "${DB_ROOT_PASSWORD}"
            MYSQL_DATABASE: "${DB_NAME}"
            MYSQL_USER: "${DB_USERNAME}"
            MYSQL_PASSWORD: "${DB_PASSWORD}"

    phpmyadmin:
        image: phpmyadmin/phpmyadmin
        ports:
            - 8080:80
        networks:
            - database
        depends_on:
            - mysql
        environment:
            PMA_HOST: mysql

We basically defined five environment variables: PROJECT_ROOT, DB_ROOT_PASSWORD, DB_NAME, DB_USERNAME, and DB_PASSWORD. We then added the corresponding placeholders in the Compose file, following the "${PLACEHOLDER}" format.

The choice was made based on the variability of these properties, in relation to the fact their values could change from one environment to another.

This allows to make the docker-compose.yml file more portable and shareable, say across a development team. You could commit a .env.example file containing placeholders for the different variables, for others to copy and rename into a .env file and change the values at will.
This way, if a developer needs a different configuration for some reason, they can do so without touching the Compose file (this is basically part of the Twelve-Factor App methodology).

In our case, it also permits to avoid repeating the configuration value for our project root, which we introduced in step 4.

See the documentation for more info about variable substitution in general.

Wrapping up

In 6 steps, we upgraded our docker-compose.yml file to the latest versions of Docker and Docker Compose, making our project simpler and better structured.

For the complete version of the file or if you had any issues while following this tutorial, please refer to the companion repository.

Cleaning up

Another nice addition of Docker 1.13.0 worth mentioning in the context of this post are clean-up commands:

$ docker system df
$ docker system prune

Clean-up

They allow to know exactly what resources Docker is using and greatly simplify freeing them, when removing dangling volumes used to require the rather esoteric docker volume rm $(docker volume ls -qf dangling=true), for instance.

It is also possible to prune a specific type of resource, e.g. for volumes:

$ docker volume prune

Conclusion

A year can be a long time when it comes to the tech industry, and Docker's evolution certainly is on a fast track. The direction it has taken is quite comforting, focussing on promoting industry standards which is essential to a technology's adoption and long term survival.

New features and ameliorations were not forgotten along the way, and using Docker and Docker Compose now feels simpler and more standardised.

While there is still room for improvement, Docker is not the obscure technology it used to be, and it is being ever more widely adopted by the industry.

It is safe to say that Docker is definitely a string to consider adding to your bow.

Sources

Posted by osteel on the :: [ docker php mysql phpmyadmin nginx tutorial webdevelopment environment ]

Comments