Docker and OpenSSH shenanigans

In Intraway we’re using Docker containers for many things, one of them is as build and testing machines for different RHEL versions. That not only requires setting up the build environment (which is simple, since we have a local yum repository where we get all build dependencies), but also checking out the source code from git and svn.

Authentication is done using ssh, and in most cases, one would use ssh keys to avoid typing passwords repeatedly. That means that the docker container needs to have access to the public and private keys inside the ~/.ssh/ folder. Obviously, copying the keys into the Dockerfiles repository is a no go for many reasons, so my approach is to mount my .ssh folder as a volume in the docker run command:

docker run –rm -it -v /home/gsanchez/.ssh/:/home/intraway/.ssh/ my_docker_image bash

There is a caveat: for this to work, one needs the my_docker_image image to have the intraway user already created with has the same uid and gid as the user in the host. In a typically installed Ubuntu, as is my case, the gid and uid is 1000, that way there will be no permissions problem.

Now, right after the docker run command, I will be staring at my containerized bash prompt ready to execute git and svn commands against the intraway servers without being asked for my login credentials. Neat.

And all will be fine until you mess with the wrong versions of OpenSSH concurrently. There is a chance you will face with the following message while trying to make an outgoing connection right before returning to the shell prompt:

[intraway@04c966bff723 tmp]$ ssh gsanchez@another_server

control_client: msg_recv

[intraway@04c966bff723 tmp]$

Huh? Let me try it again, more verbosely

[intraway@04c966bff723 tmp]$ ssh -vvv gsanchez@another_server

OpenSSH_4.3p2, OpenSSL 0.9.8e-fips-rhel5 01 Jul 2008

debug1: Reading configuration data /home/intraway/.ssh/config

debug1: Reading configuration data /etc/ssh/ssh_config

debug1: Applying options for *

debug1: auto-mux: Trying existing master

debug3: ssh_msg_send: type 1

debug3: ssh_msg_recv entering

control_client: msg_recv

[intraway@04c966bff723 tmp]$

You’ll be able to reproduce this if you have your host’s OpenSSH configured to enable ControlMaster and ControlPath is  set to ~/.ssh/something, that way, OpenSSH creates a socket inside the shared .ssh folder and the dockerized ssh instance will try to use it and avoid creating a new connection. In my case, the shared connection worked when running a dockerized RHEL6 inside an Ubuntu 14.04 host, but it failed (as described above) when running the dockerized RHEL5’s OpenSSH.

The possible workarounds are:

• Disable ControlMaster in the host’s ~/.ssh/config file. Which is overkill. And sharing TCP connections in SSH is a nice feature.
• Change the ControlPath to save the sockets outside of the shared ~/.ssh/ directory. Which is better but forces everyone that want’s to use this docker to modify this variable.
• Set an alias directly in the Dockerfile, so that ssh automatically ignores the ControlPath. This is the one I find the best option:

alias ssh=”ssh -S none”