Building docker images with private python packages

Installing private python packages into Docker container can be tricky because container does not have access to private repositories and you do not want to leave trace of private ssh key in docker…

Posted on April 24, 2018 in docker, django, python.

Installing private python packages into Docker container can be tricky because container does not have access to private repositories and you do not want to leave trace of private ssh key in docker images.

This article presents two methods that can be used.

First is using intermediate docker image where private SSH key is deployed to intermediate image that is deleted afterwards. This method can be used for other purposes where access to private server is needed only during build time.

Second method utilizes deploy or access keys that can set for various source code hosting services such as Github or Bitbucket. Idea is to create key based read only access to repositories per specific need. This access can be removed later if needed.

Check docker-images-with-private-python-packages-example repository for example files.

1. Build docker image with intermediate image

This method passes private ssh key to intermediate docker image that would be deleted after creating image.

Python packages are downloaded into intermediate container and then copied and installed in final image. It would not be possible to access private repository from inside container.

Excerpt from Dockerfile

FROM python:3.6 as intermediate
COPY requirements.txt /
WORKDIR /pip-packages/
RUN pip download -r /requirements.txt
FROM python:3.6
WORKDIR /pip-packages/
COPY --from=intermediate /pip-packages/ /pip-packages/
RUN pip install --no-index --find-links=/pip-packages/ /pip-packages/*

Description of Dockerfile:

  1. create intermediate image
  2. download python packages to intermediate image
  3. create new image
  4. copy python packages from intermediate image
  5. install downloaded packages

To create image following command can be used:

docker build --force-rm -t test-multi-stage-builds --build-arg SSH_PRIVATE_KEY="$(cat ~/.ssh/id_rsa)" .

Description of command arguments:

--force-rm - forces deleting intermediate images, even if build fails
-t test-multi-stage-builds - image name
--build-arg SSH_PRIVATE_KEY="$(cat ~/.ssh/id_rsa)" - set build-time variable so private repository can be accessed from intermediate image

2. Build docker image with deploy keys

This method depends on adding deploy/access key to every private repository. It would be possible to access repository from container until key is deleted.

To create deploy private/public keys, following command can be used:

ssh-keygen -t rsa -b 4096 -C "test-with-deploy-keys" -f "./deploy_key" -N ""

This command would create two files: private deploy_key and public deploy_key.pub. deploy_key.pub should be added to repository Settings > Deploy keys on GitHub repository settings or Settings > Access keys on Bitbucket.

docker build -t test-with-deploy-keys --build-arg SSH_PRIVATE_KEY="$(cat ./deploy_key)" -f Dockerfile-deploykeys .

Notes:

  • deploy_key would be visible in docker history and in container.
  • with this approach no intermediate image is needed.

docker-compose example

Include arguments under build key, for example:

build:
        context: .
        args:
            - SSH_PRIVATE_KEY

And here is an example of passing private key to docker-compose build.

docker-compose build --build-arg SSH_PRIVATE_KEY="$(cat ~/.ssh/id_rsa)"

Links

Github repository with all files mentioned in this article: docker-images-with-private-python-packages-example

Inspiration and initial code from :

Share on Reddit