Configuring Docker Containers

February 21, 2017

Introduction

At Intraway we have started using Docker quite a bit, especially for testing purposes. For those who don’t know, Docker images come with everything installed and ready to use. Some behavior can be altered by changing the command or mounting volumes. For instance, if we have a web server image, we can serve our site by mounting some directory into the /var/www/html directory. However, changing how Apache runs is not that easy.

There is a common approach to solving this problem in the Docker world and we use it to change the behavior of the containers in the testing environment.

Starting point

Let’s consider the following Dockerfile:

FROM ubuntu:16.04

RUN apt-get -yqq update && 
    apt-get install -yqq default-jre wget

RUN wget -O /tmp/activemq.tar.gz http://mirrors.nxnethosting.com/apache/activemq/5.14.0/apache-activemq-5.14.0-bin.tar.gz && 
    tar -xzf /tmp/activemq.tar.gz -C /opt/ && 
    rm -f /tmp/activemq.tar.gz

ENTRYPOINT ["/opt/apache-activemq-5.14.0/bin/activemq", "console"]

Then we could build it like this:

docker build -t="ndascanio/activemq:5.14.0" .

When we run the container, we use it “as is”. That means: default configuration, default user/password, default ports, etc:

docker run -d --name activemq ndascanio/activemq:5.14.0

This may be fine in most cases. However, sometimes we need to change how the service behaves. For example, what if we want to change the admin password?

A poor approach

There is a quick and easy approach. A really bad approach (but it works!). We could mount a file with the configuration we want to use to replace ActiveMQ’s default.

# File: jetty-realm.properties
admin: newpassword, admin
user: user, user

And then run the container like this:

docker run -d --name activemq -v $(pwd)/jetty-realm.properties:/opt/apache-activemq-5.14.0/conf/jetty-realm.properties ndascanio/activemq:5.14.0

Here we are replacing the jetty-realm.properties file inside the container with our custom file.

NEVER do this. EVER.

Why not?

There are several reasons:

You need to know ActiveMQ internals. You need to know which files to change and how to change them.
You are replacing the whole file, not just the admin password. As such, you may be changing more than intended.
You need to be familiar with the path where the file is. If you make a new image with ActiveMQ 5.15.0, the path changes.
Between one version and another, the internal configuration may change.

A better approach

As we are building our image, we know what is usually desirable to configure or change. We can provide some kind of “API” to do this.

A very simple and widespread solution for configuring the container is to use environment variables. Let’s take a look at the following Dockerfile:

FROM ubuntu:16.04

RUN apt-get -yqq update && 
    apt-get install -yqq default-jre wget

RUN wget -O /tmp/activemq.tar.gz http://mirrors.nxnethosting.com/apache/activemq/5.14.0/apache-activemq-5.14.0-bin.tar.gz && 
    tar -xzf /tmp/activemq.tar.gz -C /opt/ && 
    rm -f /tmp/activemq.tar.gz

ADD init.sh /app/init.sh

ENTRYPOINT ["/app/init.sh"]

It’s almost the same but we added a init.sh file:

#!/bin/bash

ACTIVEMQ_BASE=/opt/apache-activemq-5.14.0
JETTY_REALM_FILE=$ACTIVEMQ_BASE/conf/jetty-realm.properties
CONF_FILE=$ACTIVEMQ_BASE/conf/activemq.xml

if [ ! -z "$ADMIN_PASSWORD" ];then
    sed -i "s/^admin:.*/admin: $ADMIN_PASSWORD, admin/" $JETTY_REALM_FILE
fi

if [ ! -z "$USER_PASSWORD" ];then
    sed -i "s/^user:.*/user: $USER_PASSWORD, user/" $JETTY_REALM_FILE
fi

if [ ! -z "$PERCENT_JVM_HEAP" ];then
    sed -i "s#<memoryUsage percentOfJvmHeap=.*#<memoryUsage percentOfJvmHeap="$PERCENT_JVM_HEAP" />#" $CONF_FILE
fi

if [ ! -z "$STORE_USAGE" ];then
    sed -i "s#<storeUsage limit=.*#<storeUsage limit="$STORE_USAGE" />#" $CONF_FILE
fi

if [ ! -z "$TEMP_USAGE" ];then
    sed -i "s#<tempUsage limit=.*#<tempUsage limit="$TEMP_USAGE" />#" $CONF_FILE
fi

/opt/apache-activemq-5.14.0/bin/activemq console

This file checks whether some environment variables aren’t empty, and changes the configuration accordingly. After making all the changes, it starts ActiveMQ.

If we run the container with the same command as before we will have the same behavior. However, if we want to change a configuration we can run it while defining some of the environment variables like this:

docker run -d --name activemq -e ADMIN_PASSWORD=newpass -e USER_PASSWORD=hello  ndascanio/activemq:5.14.0

The init.sh can be as complex as you like. However, we recommend to keep it as simple as possible, that way you only change what’s really important.

Benefits

There are several benefits to this approach:

The user doesn’t need to know the internals of the image to make configuration changes.
The user can change between one version and the next without changing anything else.
From the user’s point of view, configuration is really easy.
All the complexity is kept within the image.

Changing configuration on the fly

What we have seen so far is how to change the default configuration of a container when it starts. Generally this should be enough. However, when making automated tests in a CI server, we need to change some things inside of a container that’s already running. The best solution is for the application or service running inside the container to provide an API to change it.

We have come up with this solution to change containers on the fly (ONLY for testing environments where the application doesn’t have an API to manipulate it).

First we’ve developed a simple generic REST server that can be configured with a yaml file and execute commands accordingly.

Continuing with the ActiveMQ example, let’s take a look at the new Dockerfile:

FROM ubuntu:16.04

RUN apt-get -yqq update && 
    apt-get install -yqq default-jre wget

RUN wget -O /tmp/activemq.tar.gz http://mirrors.nxnethosting.com/apache/activemq/5.14.0/apache-activemq-5.14.0-bin.tar.gz && 
    tar -xzf /tmp/activemq.tar.gz -C /opt/ && 
    rm -f /tmp/activemq.tar.gz

ADD class /opt/class/class
ADD activemq.yaml /opt/class/activemq.yaml

ENTRYPOINT ["/opt/class/class", "/opt/class/activemq.yaml"]

And the activemq.yaml file:

server:
    ip: '0.0.0.0'
    port: 9090

commands:
    start_activemq:
        command: '/opt/apache-activemq-5.14.0/bin/activemq start'
        params: []

    stop_activemq:
        command: '/opt/apache-activemq-5.14.0/bin/activemq stop'
        params: []

    change_admin_passwd:
        command: 'sed -i "s/^admin:.*/admin: {password}, admin/" /opt/apache-activemq-5.14.0/conf/jetty-realm.properties'
        params: []

files:
    log_activemq: '/opt/apache-activemq-5.14.0/data/activemq.log'

The Dockerfile adds the REST server (claas) and the configuration file (activemq.yaml). The entrypoint runs the server.

With the configuration file we specify the command and file endpoint. In this example we have 3 command endpoints and 1 file endpoint:

<host>:9090/command/start_activemq (POST)
<host>:9090/command/stop_activemq (POST)
<host>:9090/command/change_admin_passwd (POST)
<host>:9090/file/log_activemq (GET)

Example

curl -X POST '<activemq host>:9090/command/change_admin_passwd' -d 'password=newpassword'
curl -X POST '<activemq host>:9090/command/start_activemq'
curl -X GET '<activemq host>:9090/file/log_activemq'

First, we change the admin password with the command/change_admin_passwd endpoint. We specify the password in the body of the POST message. Then, we start activemq with the command/start_activemq enpoint. Finally we get the log file with the file/log_activemq endpoint.

Security issues

As you can see, this simple server is completely insecure and must NEVER be used in production, for the following reasons:

It doesn’t have any authentication.
It doesn’t sanitize variables.
Command injection is really easy.
Runs as root!

However, for isolated testing purposes it is quite useful.

Conclusion

We have seen how to change the configuration inside a container when the application or service doesn’t have an API. This can make our images highly customizable, thus avoiding having to rebuild the images for different purposes. Also customizations can be done easily, and the user has to know very little about the application or service.

Lastly, we have seen how to change a container on the fly. Keep in mind however that this is not recommended in production environments because of the security implications.