matthewfeickert/intro-to-docker repository overview

⁠Intro to Docker

An opinionated introduction to using Docker⁠ as a software development tool

⁠Table of Contents

• Requirements⁠
  • Installation⁠
• Documentation⁠
• Tutorial⁠
  • Docker Images and Containers⁠
  • Docker Hub⁠
  • Pulling Images⁠
  • Running Containers⁠
  • Monitoring Containers⁠
  • Exiting and restarting containers⁠
  • File I/O with Containers⁠
    • Copying⁠
    • Volume mounting⁠
  • Running Jupyter from a Docker Container⁠
  • Docker as Containers as a Service (CaaS)⁠
  • Continuous Integration (CI) and Continuous Deployment (CD)⁠
  • Dockerfiles⁠
• Contributing⁠
• Authors⁠

⁠Requirements

• A computer with Docker installed on it

⁠Installation

To install Docker Community Edition on your Linux, Mac, or Windows machine follow the instructions in the Docker docs⁠.

⁠Documentation

The official Docker documentation and tutorial⁠ can be found on the Docker website. It is quite thorough and useful. It is an excellent guide that should be routinely visited, but the emphasis of this introduction is on using Docker, not how Docker itself works.

⁠Tutorial

A note up front, Docker has very similar syntax to Git and Linux, so if you are familiar with the command line tools for them then most of Docker should seem somewhat natural (though you should still read the docs!).

⁠Docker Images and Containers

It is still important to know what Docker is and what the components of it are. Docker images are executables that bundle together all necessary components for an application or an environment. Docker containers⁠ are the runtime instances of images — they are images with a state.

Importantly, containers share the host machine's OS system kernel and so don't require an OS per application. As discrete processes containers take up only as much memory as necessary, making them very lightweight and fast to spin up to run.

⁠Docker Hub

Much like GitHub allows for web hosting and searching for code, Docker Hub⁠ allows the same for Docker images. Hosting and building of images is free for public repositories⁠ and allows for downloading images as they are needed. Additionally, through integrations with GitHub and Bitbucket, Docker Hub repositories can be linked against Git repositories so that automated builds of Dockerfiles on Docker Hub⁠ will be triggered by pushes to repositories.

⁠Pulling Images

To begin with we're going to pull⁠ down the Docker image we're going to be working in for the tutorial

docker pull matthewfeickert/intro-to-docker

and then list the images⁠ that we have available to us locally

docker images

If you have many images and want to get information on a particular one you can apply a filter, such as the repository name

docker images matthewfeickert/intro-to-docker

⁠Running Containers

To use a Docker image as a particular instance on a host machine you run⁠ it as a container. You can run in either a detached or foreground⁠ (interactive) mode.

Run the image we pulled as an interactive container

docker run -it matthewfeickert/intro-to-docker:latest /bin/bash

You are now inside the container in an interactive bash session. Check the file directory

pwd

revealing that you are in /home/docker/data and check the host to see that you are not in your local host system

hostname

Further, check the os-release to see that you are actually inside a release of Debian (given the Docker Library's Python image⁠ Dockerfile choices)

cat /etc/os-release

⁠Monitoring Containers

Open up a new terminal tab on the host machine and list the containers that are currently running⁠

docker ps

Notice that the name of your container is some randomly generated name. To make the name more helpful, rename⁠ the running container

docker rename <CONTAINER ID> my-example

and then verify it has been renamed

docker ps

⁠Exiting and restarting containers

As a test, create a file in your container

touch test.txt

In the container exit at the command line

exit

You are returned to your shell. If you list the containers you will notice that none are running

docker ps

but you can see all containers that have been run and not removed with

docker ps -a

To restart your exited Docker container start⁠ it again and then attach⁠ it to your shell

docker start <CONTAINER ID>
docker attach <CONTAINER ID>

docker start <NAME>
docker attach <NAME>

Notice that your entry point is still /home/docker/data and then check that your test.txt still exists

cd
ls

So this shows us that we can exit Docker containers for arbitrary lengths of time and then return to our working environment inside of them as desired.

docker run --rm -it <IMAGE> /bin/bash

⁠File I/O with Containers

⁠Copying

Copying⁠ files between the local host and Docker containers is possible. On your local host find a file that you want to transfer to the container and then

docker cp <file path> <CONTAINER ID>:/

and then from the container check and modify it in some way

echo "This was written inside Docker" >> example_file.txt

and then on the local host copy the file out of the container

docker cp <CONTAINER ID>:/example_file.txt .

and verify if you want that the file has been modified as you wanted

tail example_file.txt

⁠Volume mounting

What is more common and arguably more useful is to mount volumes⁠ to containers with the -v flag. This allows for direct access to the host file system inside of the container and for container processes to write directly to the host file system.

docker run -v <path on host>:<path in container> <image>

For example, to mount your current working directory on your local machine to the data directory in the example container

docker run --rm -it -v $PWD:/home/docker/data matthewfeickert/intro-to-docker

From inside the container you can ls to see the contents of your directory on your local machine

ls

and yet you are still inside the container

pwd

You can also see that any files created in this path in the container persist upon exit

touch created_inside.txt
exit
ls *.txt

This I/O allows for Docker images to be used for specific tasks that may be difficult to do with the tools or software installed on only the local host machine. For example, debugging problems with software that arise on cross-platform software, or even just having a specific version of software perform a task (e.g., using Python 2 when you don't want it on your machine, or using a specific release of TeX Live⁠ when you aren't ready to update your system release).

⁠Running Jupyter from a Docker Container

You can run a Jupyter server from inside of your Docker container. First run a container while exposing⁠ the container's internal port 8888 with the -p flag

docker run --rm -it -p 8888:8888 matthewfeickert/intro-to-docker /bin/bash

Then start a Jupyter server⁠ with the server listening on all IPs

jupyter notebook --allow-root --no-browser --ip 0.0.0.0

though for your convince the example container has been configured with these default settings so you can just run

jupyter notebook

Finally, copy and paste the following with the generated token from the server as <token> into your web browser on your local host machine

http://localhost:8888/?token=<token>

You now have access to Jupyter running on your Docker container.

⁠Docker as Containers as a Service (CaaS)

If Docker is run in a detached⁠ state then the container will exit as soon as it has executed the commands given to it. If the clean up⁠ option is given as well, then the container will be removed once it exits.

docker run --rm matthewfeickert/intro-to-docker:latest /bin/bash -c 'cat /etc/os-release && echo "hello" && python --version'

A more practical example would be to run a Docker container with TeX Live⁠ in a repo with a LaTeX document with an entrypoint⁠ command to build the document

docker run --rm -v $PWD:/home/docker matthewfeickert/latex-docker:latest

This allows for containers to serve as either full pieces of infrastructure (e.g., reana⁠) or as individual instances that are spun up, used, and spun down (think something somewhat along the lines of AWS Lambda).

⁠Continuous Integration (CI) and Continuous Deployment (CD)

To be added for a future extension

⁠Dockerfiles

Docker images are built through the Docker engine by reading the instructions from a Dockerfile⁠. These text based documents provide the instructions though an API similar to the Linux operating system commands to execute commands during the build. The Dockerfile for the example image⁠ being used is an example of some simple extensions of the official Python 3.6.8 Docker image⁠.

As a very simple of extending the example image into a new image create a Dockerfile

touch Dockerfile

and then write in it the Docker engine instructions to add cowsay⁠ and scikit-learn⁠ to the environment

# Dockerfile
FROM matthewfeickert/intro-to-docker:latest
USER root
RUN apt-get -qq -y update && \
    apt-get -qq -y upgrade && \
    apt-get -qq -y install cowsay && \
    apt-get -y autoclean && \
    apt-get -y autoremove && \
    rm -rf /var/lib/apt-get/lists/* && \
    ln -s /usr/games/cowsay /usr/bin/cowsay
RUN pip install --no-cache-dir -q scikit-learn
USER docker

Then build⁠ an image from the Dockerfile and tag it with a human readable name

docker build -f Dockerfile -t extend-example:latest --compress .

You can now run the image as a container and verify for yourself that your additions exist

⁠Contributing

If you would like to contribute please read the CONTRIBUTING.md⁠ and then open up an Issue or a PR based on its recommendations. Contributions are welcome and encouraged!

⁠Authors

• Matthew Feickert⁠