Scale Selenium Grid in 5 Seconds with Zero Docker Experience

usability testing

The reason to use CoreOS as a Docker server is because CoreOS is an extremely light weight, stripped down Linux distribution containing none of the extras that are associated with Ubuntu and the like. And it was designed to run Docker and Docker clusters, so by using it we are buying the future.

Many cloud service provider already has CoreOS image to start with. If you are not going to be running the CoreOS server on bare metal or if you have other Docker server already installed, you can skip this step.

Install CoreOS on Bare Metal (Hardware)

Download the stable CoreOS ISO from here: Download Link

Then you can burn the ISO into a CD/DVD or a bootable USB disk and use it as the boot source to boot up your physical server.

Once the command line becomes available, create a cloud-config.yml file with the ssh-key you will be using to connect to the server from remote later. The content of the file should look like this:

#cloud-config

ssh_authorized_keys:
  - ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0g+ZTxC7weoIJLUafOgrm+h...

CoreOS allows you to declaratively customize various OS-level items, such as network configuration, user accounts, and systemd units. This document describes the full list of items we can configure. The coreos-cloudinit program uses these files as it configures the OS after startup or during runtime.

Unlike AWS, cloud-config file is run during EACH system boot. While it is inspired by the cloud-init project, there are tools isn’t used by CoreOS. Only the relevant subset of its configuration items is implemented in CoreOS cloud-config file. Please refer to the official CoreOS document for detial on the subject.
cloud-config Documentation Link

Now run the install command with the cloud-config file name as argument:

coreos-install -d /dev/sda -c cloud-config.yml

Once you complete the above steps, reboot the server and make sure you remove your boot disk or boot CD/DVD. The server will boot up in command line mode and display it’s IP address obtained from DHCP server. From this point on, you can only ssh into the server via the ssh-key you provided earlier.

Install and Setup Docker Compose

Compose is a tool for defining and running multi-container applications with Docker. With Compose, you define a multi-container application in a single file, then spin your application up in a single command which does everything that needs to be done to get it running.

Docker Compose is the key component here to make spinning up or tearing down an entire Selenium Grid Farm (1 hub + multiple browsers) in a single command. Yes, you read it right, just ONE command. Installing Compose is a bit tricky though because of how CoreOS is built.

Here is the video that showed me how to install docker-compose on CoreOS toward the end of the video. You can skip the video because it’s 46 minutes long. And I will show you the exact steps on how you can do the same next.

Install Docker Compose

docker-compose is just a precompiled binary file we can download from its Github Page. Check out the link and choose the latest version number to replace in the following code (run as the default core user):

mkdir ~/bin
curl -L https://github.com/docker/compose/releases/download/1.3.3/docker-compose-`uname -s`-`uname -m` > ~/bin/docker-compose
chmod +x ~/bin/docker-compose
echo "export PATH="$PATH:$HOME/bin"" >> ~/.bashrc
source ~/.bashrc

Again if you are not using CoreOS, please follow the official installation instruction and skip the above code block.

Now the docker-compose command is ready for use, simply run docker-compose in shell, you should see the following:

core@localhost ~ $ docker-compose
Define and run multi-container applications with Docker.

Usage:
  docker-compose [options] [COMMAND] [ARGS...]
  docker-compose -h|--help

Options:
  -f, --file FILE           Specify an alternate compose file (default: docker-compose.yml)
  -p, --project-name NAME   Specify an alternate project name (default: directory name)
  --verbose                 Show more output
  -v, --version             Print version and exit

Commands:
  build              Build or rebuild services
  help               Get help on a command
...

Configure Docker Compose Application File

We’ll be using Docker images build by the official Selenium repository on Docker Hub, so you don’t need to build your own.

Under /home/core/ , create a file named docker-compose.yml with the following content:

hub:
  image: selenium/hub
  ports:
    - "4444:4444"
firefox:
  image: selenium/node-firefox
  links:
    - hub
chrome:
  image: selenium/node-chrome
  links:
    - hub

You can use other browser images in the Selenium repository in the docker-compose.yml file, and they should just work.

Manage Selenium Grid with Docker Compose

If you are running this for the first time, expect Docker to download all the necessary images in the beginning. Once all the images are downloaded into local repository, it will just take seconds to start the Selenium Grid in the future.

Start Selenium Grid

As user core, run:

docker-compose -f ~/docker-compose.yml up -d

By appending the -d argument at the end of the command, you are telling Compose to run the application in the background (daemon).

Sample output:

core@localhost /usr $ docker-compose -f ~/docker-compose.yml up -d
Recreating core_hub_1...
Recreating core_firefox_1...
Recreating core_chrome_1...

And you can verify it by running docker ps:

core@localhost /usr $ docker ps
CONTAINER ID        IMAGE                          COMMAND                CREATED              STATUS              PORTS                                                                         NAMES
a09d3ab302fa        selenium/node-chrome:latest    "/opt/bin/entry_poin   About a minute ago   Up About a minute                                                                                 core_chrome_1
298a612a387a        selenium/node-firefox:latest   "/opt/bin/entry_poin   About a minute ago   Up About a minute                                                                                 core_firefox_1
62136ab8fdb0        selenium/hub:latest            "/opt/bin/entry_poin   About a minute ago   Up About a minute   0.0.0.0:4444->4444/tcp                                                        core_hub_1

To see the real action in browser, open url http://your-coreos-IP:4444/grid/console

enter image description here