Handy Docker Scripts
Introduction:
Here we go, yet another docker post!
I'm going to keep this one short and simply use it as a repository to store some bash scripts I've written to automate some docker functions.
Scripts:
Some of these are rather hacked together, they generally work but may need some tweaking here and there!
Clean Images:
This script removes redundant images which are marked as dangling by the docker engine.
#!/bin/bash
#Author: David Chidell
echo '***Removing redundant images...'
IMAGES=$(docker images --filter dangling=true -q)
docker rmi $IMAGES > /dev/null 2>&1
Removed unused images:
This script compares the output of docker ps -a and docker images and finds images which have been downloaded but not used for any existing containers, then removes the containers which are unused. Useful for cleaning up after playing around.
#!/bin/bash
#Author: David Chidell
echo '***Removing unused images...'
used_images=$(docker ps -a | tail -n +2 | awk {'print $2'} | sort | uniq)
all_images=$(docker images | grep -v REPOSITORY | cut -d' ' -f1 | sort | uniq)
unused_images=$(diff <(echo "$all_images") <(echo "$used_images") | grep '<' | awk {'print $2'})
docker rmi $unused_images 2>&1
Update containers / apps:
This one is a little more full-on. It's also rather insecure but we don't worry about things like that!
This script performs the following:
- Downloads the latest image for running containers
- Finds container definitions contained within the containers directory.
- Deletes the old container
- Re-creates the new container (more on container definitions later)
- Starts the new container
- Finds app definitions (based on docker-compose) contained within the apps directory.
- Deletes all containers part of the app
- Re-creates the containers based on the docker-compose file
- Starts the app
- Deletes old redundant images
#!/bin/bash
#Author: David Chidell
echo '***Updating Images...'
#This line updates ALL images in the repo
#docker images | grep -v REPOSITORY | cut -d' ' -f1 | xargs -L1 docker pull
#This one only updates images we're using.
docker ps | tail -n +2 | awk {'print $2'} | sort | uniq | xargs -L1 docker pull
echo '***Updating Containers...'
CONTAINERS=$(ls containers/ | grep .txt)
for container in $CONTAINERS
do
container_name=$(echo $container | cut -d'.' -f1)
echo '*Stopping and removing ' $container_name
docker stop $container_name > /dev/null 2>&1
docker rm $container_name > /dev/null 2>&1
echo '*Creating and starting ' $container_name
docker start $(bash containers/$container) > /dev/null 2>&1
done
echo '***Updating Applications...'
DIR=$pwd
APPS=$(ls apps/)
for app in $APPS
do
cd apps/$app
docker-compose stop
docker-compose rm -f
docker-compose up -d
cd $DIR
done
#Cleanup redundant images
bash $DIR/clean_images.sh
Here's a python version as well (bit longer / more robust):
https://github.com/dchidell/docker_manager
import docker
import argparse
import os
import subprocess
container_dir = 'containers/'
app_dir = 'apps/'
def parse_args():
parser = argparse.ArgumentParser()
parser.add_argument("-u",help="Update container images",action="store_true")
parser.add_argument("-r",help="Redeploy existing containers / apps",action="store_true")
parser.add_argument("-p",help="Prune / delete stopped containers",action="store_true")
parser.add_argument("-d",help="Delete redundant images",action="store_true")
return parser.parse_args()
def return_txt_file_names(dir):
files = os.listdir(dir)
file_names = [file.split(".")[0] for file in files if '.txt' in file]
return file_names
def exec_command(cmd,success,fail="",failcare=True):
cmdlist = cmd.split(" ")
proc = subprocess.Popen(cmdlist,stdout=subprocess.PIPE,stderr=subprocess.PIPE)
err = proc.stderr.read().decode("UTF-8")
out = proc.stdout.read().decode("UTF-8")
if failcare:
if err is "":
print(success)
if out:
return out
else:
return True
else:
print("{} Reason: {}".format(fail,err))
return False
else:
print(success)
return out
def find_unused_images(all_images,all_containers):
used_images = []
for image in all_images:
for container in all_containers:
if image.attrs['Id'] == container.attrs['Image']:
used_images.append(image)
break
unused_images = [image for image in all_images if image not in used_images]
return unused_images
def get_image_name(image):
if image.attrs['RepoTags'] == '' or image.attrs['RepoTags'] is None:
if image.attrs['RepoDigests'] == '':
return 'ID: '+image.attrs['Id']
else:
return ''.join(image.attrs['RepoDigests']).split('@')[0]
else:
return ''.join(image.attrs['RepoTags'])
def main():
client = docker.from_env(version="auto",timeout=30)
args = parse_args()
argcount = 0
for arg in vars(args):
if getattr(args,arg):
argcount += 1
if argcount == 0:
print("***No Arguments detected, running everything...")
if args.p or argcount == 0:
pruned = client.containers.prune()
if pruned['ContainersDeleted'] == None:
print("***No containers to be deleted")
else:
print("***{} containers deleted. {} bytes of space reclaimed.".format(pruned['ContainersDeleted'].len(),pruned['SpaceReclaimed']))
if args.u or argcount == 0:
images = client.images.list()
count = 0
for image in images:
if image.attrs['RepoTags']:
name = get_image_name(image)
print("***Updating image "+name)
try:
client.images.pull(name)
except docker.errors.ImageNotFound:
print("***Image not found in docker repo! Unable to update.")
else:
print("***Updated "+name)
count += 1
print("***Update complete. Processed {} images.".format(count))
if args.r or argcount == 0:
files = os.listdir(container_dir)
container_names = [file.split(".")[0] for file in files if '.txt' in file]
containers = client.containers.list(all=True)
for container in containers:
if container.name in container_names:
print("***Stopping and deleting container: "+container.name)
container.stop()
container.remove()
print("***Re-creating container: "+container.name)
out = exec_command("bash {}{}.txt".format(container_dir,container.name),"***Container created successfully! Starting up...","***Error detected")
if out:
new_container = client.containers.get(out.strip())
new_container.start()
app_names = os.listdir(app_dir)
for app in app_names:
print("***Found app: "+app)
print("***Stopping app...")
exec_command("docker-compose -f {}{}/docker-compose.yml stop".format(app_dir,app),"***App stopped successfully!",failcare=False)
exec_command("docker-compose -f {}{}/docker-compose.yml rm -f".format(app_dir,app),"***App deleted successfully!",failcare=False)
exec_command("docker-compose -f {}{}/docker-compose.yml up -d".format(app_dir,app),"***App started successfully!",failcare=False)
if args.d or argcount == 0:
images = find_unused_images(client.images.list(),client.containers.list(all=True))
count = 0
for image in images:
print('***Redundant Image found: '+get_image_name(image))
try:
client.images.remove(image.attrs['Id'])
except docker.errors.APIError:
print('***Unable to remove {}. Likely it is a parent image.'.format(get_image_name(image)))
else:
count += 1
print('***{} redundant images removed.'.format(count))
main()
Container and application definitions are stored as .txt files inside the containers and apps directories respectively.
These definitions are essentially bash commands which would be used to create a container. An example of a container.txt file would be as follows:
docker create \
--name=portainer \
--restart=always \
-v /var/run/docker.sock:/var/run/docker.sock \
-p 9000:9000 \
portainer/portainer
This is generally insecure as this file is executed by bash. These files can be manipulated to contain any bash script.
A container / app can be disabled by changing the file extension from .txt to something else (Note: If the file still contains the .txt extension it will NOT be disabled!)