diff --git a/docker/README.md b/docker/README.md index 2db67a5f2..2b0f0e05b 100644 --- a/docker/README.md +++ b/docker/README.md @@ -11,3 +11,36 @@ The following table provides a list of Dockerfiles that you can use to build the | `merlin-hugectr` | `dockerfile.ctr` | | NVTabular and HugeCTR | | `merlin-tensorflow` | `dockerfile.tf` | | NVTabular, TensorFlow, and HugeCTR Tensorflow Embedding plugin | | `merlin-pytorch` | `dockerfile.torch` | | NVTabular and PyTorch | + + +## Building the Merlin Containers Locally + +Building our containers is a two-step process. We first build the Merlin `BASE_IMAGE` using the `dockerfile.merlin` file. This container depends on two upstream containers: `nvcr.io/nvidia/tritonserver` and `nvcr.io/nvidia/tensorflow`, from which it pulls the necessary dependencies for Triton Inference Server and RAPIDS tools. It also builds and installs other Merlin requirements, such as scikit-learn, XGBoost, etc. + +We then build framework-specific containers based off of the `BASE_IMAGE`. See the table above for which Dockerfile corresponds to which framework-specific container. + +The base image is not made available publicly, but the framework-specific containers based on it are. The two-stage build process takes roughly 1 hour. Running all of the tests for all Merlin libraries can take a couple of additional hours, depending on which framework you're building. + +## Building The `BASE_IMAGE` + +We tag this image as `nvcr.io/nvstaging/merlin/merlin-base:${MERLIN_VERSION}` and it is used to create the framework-specific containers. There are `ARG`s in the Dockerfile to define which version of the containers to use. You can override the defaults when building the image like below. + +```bash +docker build -t nvcr.io/mycompany/merlin-base:${MERLIN_VERSION} --build-arg TRITON_VERSION=23.04 - < docker/dockerfile.merlin +``` + +In this example we are tagging the base image as `nvcr.io/mycompany/merlin-base:${MERLIN_VERSION}`. The tag Merlin uses when building this image in our own build pipeline is `nvcr.io/nvstaging/merlin/merlin-base:${MERLIN_VERSION}`. + +## Building A Framework-Specific Container + +We also provide Dockerfiles for creating framework-specific containers: `dockerfile.tf`, `dockerfile.torch`, and `dockerfile.ctr`. These are all based on the `BASE_IMAGE` created in the previous step and install the associciated deep learning frameworks. + +To build the PyTorch container, we specify the `BASE_IMAGE` build arg to use the base image that we just created. + +```bash +docker build -t ngcr.io/mycompany/merlin-torch:${MERLIN_VERSION} --build-arg BASE_IMAGE=nvcr.io/mycompany/merlin-base:${MERLIN_VERSION} - < docker/dockerfile.torch +``` + +## Default Arguments + +Each of the Dockerfiles have many `ARG`s defined, most of which have defaults set. Sometimes the defaults fall out of date, because the Merlin team overrides them in our build process as demonstrated above. To see the `ARG` values used in each of our released containers, see the [Merlin Support Matrix](https://nvidia-merlin.github.io/Merlin/main/support_matrix/index.html)