The article outlines a method to transfer Docker container data volumes between hosts.
Recently I had to migrate different docker based projects from the old machine to the new one. The easiest thing that came to my mind was to install the same version of Docker on the new machine and copy the whole /var/lib/docker/
docker directory. This did not work.
Projects that were needed to be copied were different web applications (mostly Django and 2 php projects). All of them shared a property that they were defined in a docker-compose file, and as such were easily rebuilt on the new machine by copying directories to the new machine and running docker-compose up --no-start
for each of them. Except for the data.
Idea is to write a command that would automate the copying of data inside docker containers. Nice thing is that the container name is predictive (see note).
Here is the command that will copy data to a container on the remote machine.
(CONTAINER=myproject_db_1 REMOTE_HOST=newhost DIR=/var/lib/mysql; \
docker run --rm \
--volumes-from "$(docker inspect --format={{.Id}} $CONTAINER)" \
busybox tar -czv --to-stdout $DIR \
| ssh $REMOTE_HOST docker run --rm -i \
--volumes-from "\$(docker inspect --format={{.Id}} $CONTAINER)"\
busybox tar -xzv \
)
This command can be divided in three parts: set variables, create archive, upload and expand archive. Bellow are explanations for parts of the command so it can be understood and adapted for different needs.
CONTAINER=myproject_db_1 REMOTE_HOST=newhost DIR=/var/lib/mysql;
CONTAINER
- container name that we want to copy data from
REMOTE_HOST
- remote host we are copying data to
DIR
- location of data inside of container, ie: for MySql it is /var/lib/mysql
, and for postgres it is /var/lib/postgresql/data
docker run - rm \
--volumes-from "$(docker ps -aqf "name=$CONTAINER")" \
ubuntu tar -czv --to-stdout $DIR
run
- Run a command in a new container
--rm
- Automatically remove the container when it exits
--volumes-from
- Mount volumes from the specified container
docker inspect --format={{.Id}} $CONTAINER
- get container id by its name
busybox
- we use BusyBox docker image to run command (see Notes)
tar -czv --to-stdout $DIR
- (c)reate g(z)ipped (v)erbose archive of data directory to standard output
| ssh $REMOTE_HOST docker run --rm -i \
--volumes-from "\$(docker inspect --format={{.Id}} $CONTAINER)"\
busybox tar -xzv
|
- pipe output of previous command
ssh $REMOTE_HOST
- connect to remote host
docker run --rm
- run docker on remote host, removing the container when it exits
-i
- keep STDIN open even if not attached
--volumes-from "\$(docker inspect --format={{.Id}} $CONTAINER)"
- mount volume from remote container. Notice that dollar sign is escaped, so command substitution occurs on the remote machine and not on the local machine.
busybox tar -xzv
- e(x)tract g(z)ip archive (v)erbose.
ubuntu
or some other distribution