Automating Docker with Python

Introduction:

Ok I've gone docker mad, I love it! Everything of mine is now running on docker (at least everything I can). Naturally the next step is to bring some level of automation to the show, who doesn't love automation!

Getting Started with the API

The raw docker API is REST (mostly) and accessible from a unix socket on the host residing at /var/run/docker.sock. This is actually a rather nice interface - and we'll cover why later, however sometimes you may want to expose the API via a TCP port instead, for remote communications.

Changing the socket

You can actually change the socket inside the /etc/default/docker file which is part of the upstart script (if you read the proxy section in the previous docker post you'd remember I'm using upstart). The section you'd want to change is the DOCKER_OPTS line. Here is mine (the default):

# Use DOCKER_OPTS to modify the daemon startup options.
#DOCKER_OPTS="--dns 8.8.8.8 --dns 8.8.4.4"

We can specify a new socket file, or indeed a TCP socket to listen on.

Unix Socket:

DOCKER_OPTS="-H unix://var/run/mydirty.sock"

This changes the default /var/run/docker.sock to /var/run/mydirty.sock

TCP Socket:

DOCKER_OPTS="-H tcp://0.0.0.0:2345"

This one is a little more interesting, as it now exposes the socket via TCP, so you can remotely connect to the API. If you wish to only access the socket from a particular network interface you can specify the IP of the interface instead of 0.0.0.0

Interfacing with Python

So there's a couple of ways, and actually I'm going to gloss over one and cover the other in more detail.

The first, as you'd expect is hardcore mode. Now you know how to access the API you can do so using pure python using our good friend the requests library. If you're using a unix socket you will have to use requests_unixsocket and the URL will look something like http+unix://hostip:2345.

A very basic request (i.e. to the base URL) would look like the following:

import requests_unixsocket

#Create the session
session = requests_unixsocket.Session()

#Send the request
resp = session.get('http+unix://hostip:2345/')

This is all well and good but actually someone out there has already done the hard work for us!

This brings us to option #2. Some rather nice soul has gone to the effort to create a docker library for python! First you'll want to install it, best way has to be via pip:

pip install docker or pip3 install docker depending if you're a python 2 or 3 guy (I'm a 3 myself).

Now we can just import the library and start tinkering right away:

import docker

client = docker.from_env()

#Get a list of running containers, the same as 'docker ps'
container_list = client.containers.list()

for container in container_list:
  print('We found a container! Name: ' + container.name)

It really is that easy! The documentation for this library is fantastic and available here:
https://docker-py.readthedocs.io/en/stable/

A more complex example...

Here's something I threw together today mainly because I could, but this script tests the docker engine on a host by downloading the alpine nginx container, running 1000 instances of it, making a HTTP request to each container and confirming each instance is online. Then destroys all the containers and exits.

This took a while to run for me, and consumed about 10GB of RAM. Don't do it on a production system!

#!/usr/bin/python3

import docker
import requests

def main():

  # Connect to the default unix socket (/var/run/docker.sock)
  client = docker.from_env()

  #Pull the nginx:alpine image
  client.images.pull('nginx:alpine')

  #Define some test parameters, our first HTTP port and the number of containers
  portstart = 10000
  count = 1000

  #Create and start 'count' number of containers. Map container port 80 to an external port.
  for a in range(1,count+1):
    container = client.containers.create('nginx:alpine',ports={'80/tcp':portstart+a})
    container.start()
    print('Created container number {} name: {}'.format(a,container.name))

  #Get a list of all the running containers (best you don't run this script on a system which has existing containers running)
  #Iterate through the list, pick out the remote port, and perform a GET request to it to check nginx is online. If the status code is 200, we must be successful!
  container_list = client.containers.list()
  count = 0
  for container in container_list:
    port = container.attrs['HostConfig']['PortBindings']['80/tcp'][0]['HostPort']
    r = requests.get('http://127.0.0.1:{}'.format(port))
    if(r.status_code == 200):
      print('Container {} is alive and working on port {}!'.format(container.name,port))
      count += 1
    else:
      print('Container {} is dead :( Code: {}'.format(container.name,r.status_code))

  print('Summary: Online Containers: {} Offline Containers: {}'.format(count,len(container_list)-count))
  print('Removing containers...')
 
  #Let's clean up and put our toys back in the toybox.
  for container in container_list:
    container.stop()
    container.remove()

if __name__ == "__main__":
    main()

Voila! How easy is that!