Using Docker to Run Python

August 14, 2014 Jenny Farver

Here at Civis we like to use the best tool for the job, so integrating different tools and stacks is essential.

Docker is the tool-du-jour for portable code. It allows us to maintain a rich environment for statistical programming, and easily call that environment’s executables from our webapp.

Props to Docker for a great online tutorial. It’s available here. It’s a good way to get your hands dirty, but alas, it’s not real. If you’re looking to take the next step, try following the example I’ve shared below.

In this example, we’ll:

  • install and configure Docker,
  • use a Docker image with a Python installation inside it,
  • build a new Docker image which contains our own Python executable.

1. Install Docker.

If you’re on Linux, installation should be straightforward. I’m on Mac OS X and that’s pretty easy too. Installation instructions can be found on Docker’s website. Mac users will also need to install boot2docker, a lightweight virtual machine.

Installing Docker

Check your installation by running:

  $ docker run hello-world
Hello from Docker.

As explained in the installation instructions, this downloads the hello-world image and creates a bare-bones container.

2. Identify the Python code you’d like to run in Docker.

I want to run a simple app that demonstrates random incidence via the Birthday Problem. Rather than writing that from scratch, I found a GitHub repo here that almost does what I want.

I do want to make a few changes. Namely:

  1. I want it to be Python 3 compatible and
  2. I want to provide all of the argument when I execute it; no interactive command line input.
  3. I’d like it to write the results to a JSON blob.

To do so, I forked the repository in GitHub. These changes aren’t really the point, but you can view them here.

clone the repository:

$ git clone

3. Set up a specific Python version to run in a Docker container.

First, check to see your current system python version:

$ python --version

Mine is 2.7.6 and I want my script to run in Python 3. An easy way to do this is to pull a pre-built image that already has what you need. Shared images can be found on Dockerhub. (You’ll need to create an account to use Dockerhub, or you can use your GitHub account).

On Dockerhub, I searched ‘python’ and found this repo. That seems useful, but before I set that up, try:

  $ docker run hello-world python --version
2014/08/09 13:48:43 exec: "python": executable file not found in $PATH
2014/08/13 13:34:49 Error response from daemon: Cannot start container 8a323746b498b874ce8efe1b5a556c42f8ffe37d49930559d7eb82926e4469f2: exec: "python": executable file not found in $PATH

Python isn’t installed on the hello-world image. Now pull a docker image that does have Python 3.

$ docker pull python

Now I can run python in the container from that image. For example:

$ docker run python python --version
  Python 3.4.1

This command says “run the docker image called ‘python’ and in that container execute the command ‘python –version’.” You can see that the container has Python 3, while my own machine still has Python 2.

The ‘python’ image also has a number of other basic utilities in it, so we can get a feel of what’s there. ‘ls’ shows my file system’s directory,

$ docker run python ls

shows the directories for the python container.

4. Build an image that has your Python code.

There are a few ways to get your Python code to run in the ‘python’ container. One is to mount a local directory to that image, but like most of our Dev team here at Civis, I’m on OS X. There’s an incompatibility there — in short “bind mount can’t work on a remote machine, so if your client is on OS X and your daemon on boot2docker, -v /host/some/path:/container/some/path will look for /host/some/path on the boot2docker host, not you mac.” (see this issue for more information).

Instead, we’ll build a container that has our Python code in it. To do so, we’ll first create a Dockerfile, which contains a series of commands to build the container.

From your git Python repo, make a Dockerfile like this:


FROM python COPY . /src CMD ["python", "/src/"]

and save it as “Dockerfile”

This creates a new docker image, starting from the ‘python’ image. It copies current dir into /src for the image, then ???

  $ docker build -t jennyfar/python-birthday .

Sending build context to Docker daemon 55.3 kB Sending build context to Docker daemon Step 0 : FROM python ---> f9014fefee22 Step 1 : COPY . /src ---> b773130f5f97 Removing intermediate container 1b82602e15ac Step 2 : CMD ["python", "/src/"] ---> Running in 5bc193de7611 ---> 6cad1dc30dd5 Removing intermediate container 5bc193de7611 Successfully built 6cad1dc30dd5

The -t flag supplies a tag for the resulting image. Without it, you’ll get an image id.

Remember to make sure ‘DOCKER HOST’ is set in your environment, as specified in the Docker installation instructions.

5. Wheeeeeee!!!

Let’s test drive our new container.

  $ docker run jennyfar/python-birthday python --version
Python 3.4.1

Good. Our new container has Python 3. And now, let’s run in the new container, supplying arguments (num_people, num_successful_trials):

$ docker run jennyfar/python-birthday python /src/ 25 10

This will simulate the 'Birthday Problem' by generating XX random numbers between 1 and 365 (inclusively).

{"expected_successful_trials": "56.8700", "percent_successful_trials": "50.0000", "percent_error": "13.7399", "num_people": 25, "total_trials": 10, "num_successful_trials": 5}

The post Using Docker to Run Python appeared first on Civis Analytics.

Previous Article
Piaggio: The Accurate Way to a Story Worth Telling
Piaggio: The Accurate Way to a Story Worth Telling

Piaggio’s Question: What combination of attributes in our ads will positively impact the perception of our ...

Next Article
Measuring Pi with Lentils and Python
Measuring Pi with Lentils and Python

Here at Civis, we like to have fun. And what’s more fun than coming up with new and creative ways to measur...