Streamline your container workflow with Codefresh!

Everyday Hacks for Docker

Docker Tutorial | January 5, 2017

In this post, I’ve decided to share with you some useful commands and tools I frequently use when working with awesome Docker technology. There is no particular order or “coolness level” for every “hack.” I will simply present the use case and how the specific command or tool has helped me with my work. Read these great hacks and make sure to check out the great hack of all – Codefresh –  the best CI for Docker out there.

Docker Hacks

Cleaning Up

After working with Docker for some time, you start accumulating development junk: unused volumes, networks, exited containers and unused images.

One Command to “Rule Them All”

prune is a very useful command (also works for volume and network sub-commands), but it is only available for Docker 1.13. So if you’re using older Docker versions, the following commands can help you to replace the prune command.

Remove Dangling Volumes

dangling volumes are volumes not in use by any container. To remove them, combine two commands: first, list volume IDs for dangling volumes and then remove them.

Remove Exited Containers

The same principle works here too. First, list the containers (only IDs) you want to remove (with filter) and then remove them (consider rm -f to force remove).

Remove Dangling Images

dangling images are untagged images, that are the leaves of the images tree (not intermediary layers).

Autoremove Interactive Containers

When you run a new interactive container and want to avoid typing rm command after it exits, use --rm option. Then when you exit from the created container, it will be automatically destroyed.

Inspect Docker Resources

jqjq is a lightweight and flexible command-line JSON processor. It is like sed for JSON data. You can use it to slice and filter, and map and transform structured data with the same ease that sed, awk, grep and friends let you play with text.

docker info and docker inspect commands can produce output in JSON format. Combine these commands with jq processor.

Pretty JSON and jq Processing

Watching Containers Lifecycle

Sometimes you want to see containers being activated and exited when you run certain docker commands or try different restart policies. Thewatch command combined with docker ps can be pretty useful here. I found that the docker stats command (even with --format option) is not useful for this because it doesn’t allow you to see the same information as you can with the docker ps command.

Display a Table with ‘ID Image Status’ for Active Containers and Refresh it Every 2 Seconds

Enter into Host/Container Namespace

Sometimes you want to connect to the Docker host. The ssh command is the default option, but this option may not be available, due to security settings, firewall rules or other undocumented issues (just try googling “how to ssh into Docker for Mac VM” to see what I mean).

nsenter, by Jérôme Petazzoni, is a small and very useful tool for this use cases. The nsenter command allows you to enter into namespaces. I like to use the minimalistic (580 kB) walkerlee/nsenter Docker image.

Enter into Docker Host

You can use --pid=host to enter into Docker host namespaces.

Enter into ANY Container

It’s also possible to enter into any container with nsenter and --pid=container:[id OR name]. But in most cases, it’s better to use the standard docker exec command. The main difference is that nsenter doesn’t enter the cgroups, and therefore evades resource limitations (which can be useful for debugging).

Heredoc Docker Container

Sometimes you want to get some tool as a Docker image, but you do not want to search for a suitable image or to create a new Dockerfile (no need to keep it for future use, for example). Sometimes storing Docker image definition in a file looks like an overkill – you need to decide how you edit, store and share this Dockerfile. Sometimes it’s better to have a single line command, that you can copy, share, embed into a shell script or create special command alias.
So, when you want to create a new ad-hoc container with a single command, try a Heredoc approach.

Create Alpine Based Container with ‘htop’ Tool

Docker Command Completion

The Docker CLI syntax is very rich and constantly growing: adding new commands and new options. It’s hard to remember every possible command and option, so having a nice command completion for a terminal is a must have.

The command completion is a kind of terminal plugin, that lets you auto-complete or auto-suggest what to type in next by hitting the tab key. Docker command completion works for commands and options. The Docker team prepared command completion for the docker, docker-machine and docker-compose commands, for both Bash and Zsh  shell.

If you are using Mac and Homebrew, then installing the Docker commands completion is pretty straight forward.

If you’re not using Mac, read the official Docker documentation for install instructions: docker engine, docker-compose and docker-machine.

Start Containers Automatically

When running a process inside a Docker container,  a failure may occur due to multiple reasons. In some cases, you can fix it by re-running the failed container. If you are using a Docker orchestration engine, like Swarm or Kubernetes, the failed service will be restarted automatically.
If not, then you might want to restart the container based on the exit code of the container’s main process, or always restart (regardless the exit code). Docker 1.12 introduced the docker run command: restart for this use case.

Restart Always

Restart the redis container with a restart policy of always so that if the container exits, Docker will restart it.

Restart Container on Failure

Restart the redis container with a restart policy of on-failure and a maximum restart count of 10.

Network Tricks

There are times when you might want to create a new container and connect it to an existing network stack. This might be the Docker host network or another container’s network. This is helpful when debugging and auditing network issues.
The docker run --network/net option allows you to do this.

Use the Docker Host Network Stack

The new container will attach to the same network interfaces as the Docker host.

Use Another Container’s Network Stack

The new container will attach to the same network interfaces as the other container. The target container can be specified by id or name.

Attachable Overlay Network

Using Docker Engine running in swarm mode, you can create a multi-host overlay network on a manager node. When you create a new swarm service, you can attach it to the previously created overlay network.

Sometimes you need to attach a new Docker container (filled with different networking tools), to an existing overlay network, in order to inspect the network configuration or debug network issues.  You can use the docker run command for this, eliminating the need to create a completely new “debug service”.

Docker 1.13 brings a new option to the docker network create command: attachable. The attachable option enables manual container attachment.

 


I hope you find this post useful. If you have more tricks on your own, please leave them in the comments box!

Pssst, secret last hack, check out Codefresh for Docker native CI/CD, it’s designed specifically for Docker and can handle the entire container driven lifecycle.

 

 

Alexei Ledenev

About Alexei Ledenev

Alexei is an experienced software architect and HPE distinguished technologist. He currently works at Codefresh as the Chief Researcher, focusing lately on #docker, #golang and #aws. In his spare time, Alexei maintains a couple of Docker-centric open-source projects, writes tech blog posts, and enjoys traveling and playing with his kids. https://github.com/gaia-adm/pumba

Reader Interactions

Enjoy this article? Don't forget to share.

Comments

  1. Hi Alexei,

    Thanks for sharing.

    I’m wondering if you could share some of the commands equivalent with their APIs? meaning those docker system prune have an equivalent API or set of API calls that would do the same?

    Thanks!

    • There is no single Remote API endpoint for prune method.
      Instead, version 1.25 of Docker Remote API adds 4 new prune methods:

      POST /containers/prune prunes stopped containers
      POST /images/prune prunes unused images
      POST /volumes/prune prunes unused volumes
      POST /networks/prune prunes unused networks

  2. Might want to… s/Apline/Alpine/

    Awesome write up though! Very helpful.

  3. Shouldn’t the first one be:

    docker volume rm $(docker volume ls -qf dangling=true)
    ?

  4. Thank you for sharing your information. I really appreciate your efforts and I am waiting for your further post thanks once again.

Reply This Comment: Assaf Lavie Cancel reply

Your email address will not be published. Required fields are marked *

Follow me on Twitter