Everything you need to know about Docker to kickstart your bioinformatics project

Figure 1. Docker logo.

Introduction

Hi everyone ! Welcome to my fifth Medium post !

Just wondering have you heard of Docker? Are you interested in learning more about Docker and applying it in your day-to-day life’s bioinformatics analysis? Here’s every basic thing that you need to know about Docker to kickstart your bioinformatics project !

Why is conda installation of bioinformatics tools not always the best choice?

Oftentimes, people say that installing bioinformatics tools via conda is the easiest and safest method because it prevents you from experiencing headaches caused by installing numerous required dependencies and binaries. However, you need to be aware that the conda packages are not always the latest and updated ones, as it appears that some of the conda packages aren’t regularly maintained and updated for debugging purposes. Hence, oftentimes, it is not surprising to see that you sometimes run into an error after running the installed conda package, since the conda package that you’ve just installed isn’t well-maintained, debugged, and updated.

For instance, I’ve seen quite a number of bioinformaticians and biologists actually encountered numerous errors when they were trying to run the installed conda TransRate v1.0.3 package to evaluate their de novo transcriptome assembly. This is all due to the lack of conda package’s maintenance and updates. Therefore, I believe that bioinformaticians should not solely depend on conda packages to run bioinformatics tools. Apart from knowing how to create isolated conda environments and install bioinformatics conda packages, one should also know how to leverage the use of Docker when it comes to installation of bioinformatics tools. Trust me, knowing how to run Docker commands will definitely save you from experiencing hours of frustration, trying to get a bioinformatics tool installed !

What is Docker?

So, what is Docker? Docker is a software that is used for running and deploying containerised software applications. Docker was originally written and built for Linux (most docker containers are Linux-based). But later, an update called Docker desktop had been developed for Windows and Mac, which makes it possible to run Linux-based containers on Windows and Mac computers as well. In other words, if you wish to operate using a graphical user interface (GUI), then please feel free to download Docker desktop. However, if you are used to operating within a Linux environment, you can straight away install Docker Engine within the Linux environment and operate via Linux.

How does Docker work?

Docker works like a lightweight virtual machine system: the Docker client is what you use to run commands, which are sent to the Docker daemon (the background service that builds, runs, and manages everything). The daemon takes a Docker image (a pre-built package with code, libraries, and settings) and uses it to create a Docker container, which is a running, isolated environment where your application executes. Every time you run a Docker image, Docker creates a new container based on that image. The Docker image is like a blueprint (read-only), and the Docker container is the live, running instance of it. You can start running containers, stop the running containers, or delete containers, but the image itself will stay unchanged and reusable (unless you manually run a command to delete it via docker rmi).

Why does every bioinformatician need to know Docker? Why is it considered a powerful tool for bioinformaticians?

Each Docker image has its own file system that has everything it needs to run, for instance all of the required dependencies/binaries/system packages and etc (you don’t need to install the dependencies one by one, which is very tedious and error-prone as the docker container already contains/encapsulates all the configuration scripts, parameters, source code, packages, & dependencies in one single environment/container/package). Docker container can be easily created, shared or moved like a zip/tar/jar file. Hence, this is the reason why you no longer need to install the required dependencies, binaries and packages of the bioinformatics tools individually one by one after knowing how to use Docker. This in turn saves you a lot of time when it comes to installing bioinformatics tools.

Pre-requisite of running Docker

Bear in mind that you need to have root privileges when it comes to using Docker. This is because Docker can only run with root privileges as you need to do things like binding to privileged ports, mounting file systems and modifying network settings. Hence, if you are working on a shared high performance computing (HPC) cluster, please kindly approach the admin of the HPC cluster to assist you with the installation of Docker and remind the admin to add you (your user ID) into the Docker group, so that you will be granted permission to run Docker (otherwise, you are going to run into permission issue error while trying to run any docker-related commands). In this case, my userID is cbr14, so I just need to run the command sudo usermod -aG docker cbr14 to add the cbr14 user into the docker group.

What is Docker Hub?

Docker Hub is like a big online app store for Docker, whereby it is a cloud-based registry where developers can store, share and download Docker images. Instead of building everything from scratch, you can pull ready-made images (like databases, web servers, or tools) from Docker Hub and run them as containers, or push your own images there so others can use them.

What are the basic commands that you should know to install bioinformatics tools via Docker?

1. Search for images to check whether it is available for use in the DockerHub registry).

docker search your_search_keyword

2. Pull/download a docker image from Docker Hub).

docker pull user/image

3. List all of the currently available images that you’ve successfully pulled from Docker Hub).

docker images

4. Create and run a container in the background, where the -d flag refers to detached in this case). In this case, the name of the docker container that is running in the background is called orp_transrate, which I intentionally specified.

docker run -d --user <user_ID>:<group_ID> --name orp_transrate -v /media/raid/Wee/WeeYeZhi/output:/data user/image /bin/bash -c "commands"

5. Create and run a container in the foreground where -it refers to interactive bash mode) (Bear in mind that you cannot run the docker command with both the flags, -d and -it, specified at the same time, as one flag is for running container in the background and another flag is for running container in the foreground (live terminal).)

docker run -it --user <user_ID>:<group_ID> --name orp_transrate -v /media/raid/Wee/WeeYeZhi/output:/data user/image /bin/bash -c "commands"

6. Connect to the Docker container that is already running in the background.

docker exec -it user/image /bin/bash

7. Rename a docker container (Keep in mind, by default, if you do not specify the use of — name flag while running the docker container, a random name will be assigned to your running container.)

docker rename

8. List the process’s output log content.

docker logs name_of_the_running_docker_container

9. Output the list of running Docker containers

docker ps

10. Output the list of all the Docker containers, including running & stopped/failed ones.

docker ps -a

11. Stop the running Docker container, just in case if you spot something is going wrong and you wish to make the required correction.

docker stop <container_id> OR docker stop <container_name>

12. Restart the container after making the necessary correction.

docker restart <container_id> OR docker restart <container_name>

13. Kill the running Docker container.

docker kill <container_id> OR docker kill <container_name>

14. Remove the docker container after finished running it to avoid consuming computational resources in the background (Bear in mind that you cannot create the same docker container with the same name if you do not remove the container).

docker rm <container_id> OR docker rm <container_name>

15. Remove the docker image after finished running it to clean up disk space (Bear in mind that you need to specify the version of the docker image if its version is not the latest one while you are trying to remove the docker image, otherwise you are going to encounter an error saying that the docker image that you are trying to remove does not exist).

docker rmi <image_name_as_originally_specified_in_DockerHub>

If you’re interested to dive deep into learning Docker programming language, you can get a more detailed cheat sheet for Docker, by visiting the webpage at https://quickref.me/docker.

After you’ve got the hang of running Docker commands, rest assured that you can now effortlessly install bioinformatics tools via Docker without the need to worry about installing all the required dependencies, binaries, and packages individually. Subsequently, this will definitely save you lots of time to accelerate your bioinformatics analysis, rather than being dragged and frustrated by the unsuccessful installation of bioinformatics tools that occurred due to dependency conflicts.

Happy coding and all the very best!