Skip to content

nyuhpc/workbench-images

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

77 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Workbench Images

See changelog

Various workbench images (Jupyter, RStudio, VSCode,...) and runtimes to use with Open Data Hub (ODH) or Red Hat OpenShift Data Science (RHODS).

The runtimes images are fully in sync with their respective workbench images. So the code developed in a specific notebook image can be directly run in a pipeline using its corresponding runtime.

Runtimes can also be used as base images for packaging your code or application in a container image.

TL;DR, full list of images

NOTE: base and CUDA buildchain images are not listed here, see below for details.

Minimal Jupyter notebook - JupyterLab 3.5.3

Image description Pull address
Minimal JupyterLab Notebook image on UBI9 with Python 3.9 quay.io/opendatahub-contrib/workbench-images:jupyter-minimal-ubi9-py39_2023b_latest
Minimal JupyterLab Notebook image on CentOS Stream 9 with Python 3.9 quay.io/opendatahub-contrib/workbench-images:jupyter-minimal-c9s-py39_2023b_latest
Minimal JupyterLab Notebook image on UBI8 with Python 3.8 quay.io/opendatahub-contrib/workbench-images:jupyter-minimal-ubi8-py38_2023b_latest
CUDA 11.8 + CuDNN 8.6.0 versions:
CUDA Minimal JupyterLab Notebook image on UBI9 with Python 3.9 quay.io/opendatahub-contrib/workbench-images:cuda-jupyter-minimal-ubi9-py39_2023b_latest
CUDA Minimal JupyterLab Notebook image on CentOS Stream 9 with Python 3.9 quay.io/opendatahub-contrib/workbench-images:cuda-jupyter-minimal-c9s-py39_2023b_latest
CUDA Minimal JupyterLab Notebook image on UBI8 with Python 3.8 quay.io/opendatahub-contrib/workbench-images:cuda-jupyter-minimal-ubi8-py38_2023b_latest

Datascience Jupyter notebook

Image description Pull address
Datascience JupyterLab Notebook image on UBI9 with Python 3.9 quay.io/opendatahub-contrib/workbench-images:jupyter-datascience-ubi9-py39_2023b_latest
Datascience JupyterLab Notebook image on Centos Stream 9 with Python 3.9 quay.io/opendatahub-contrib/workbench-images:jupyter-datascience-c9s-py39_2023b_latest
Datascience JupyterLab Notebook image on UBI8 with Python 3.8 quay.io/opendatahub-contrib/workbench-images:jupyter-datascience-ubi8-py38_2023b_latest
Datascience + VSCode JupyterLab Notebook image on UBI9 with Python 3.9 quay.io/opendatahub-contrib/workbench-images:jupyter-datascience-code-server-ubi9-py39_2023b_latest
Datascience + RStudio JupyterLab Notebook image on Centos Stream 9 with Python 3.9 quay.io/opendatahub-contrib/workbench-images:jupyter-datascience-rstudio-c9s-py39_2023b_latest
CUDA 11.8 + CuDNN 8.6.0 versions:
CUDA Datascience JupyterLab Notebook image on UBI9 with Python 3.9 quay.io/opendatahub-contrib/workbench-images:cuda-jupyter-datascience-ubi9-py39_2023b_latest
CUDA Datascience JupyterLab Notebook image on Centos Stream 9 with Python 3.9 quay.io/opendatahub-contrib/workbench-images:cuda-jupyter-datascience-c9s-py39_2023b_latest
CUDA Datascience JupyterLab Notebook image on UBI8 with Python 3.8 quay.io/opendatahub-contrib/workbench-images:cuda-jupyter-datascience-ubi8-py38_2023b_latest
CUDA Datascience + VSCode JupyterLab Notebook image on UBI9 with Python 3.9 quay.io/opendatahub-contrib/workbench-images:cuda-jupyter-datascience-code-server-ubi9-py39_2023b_latest
CUDA Datascience + RStudio JupyterLab Notebook image on Centos Stream 9 with Python 3.9 quay.io/opendatahub-contrib/workbench-images:cuda-jupyter-datascience-rstudio-c9s-py39_2023b_latest

VSCode 4.9.1

Image description Pull address
VSCode image on UBI9 with Python 3.9 quay.io/opendatahub-contrib/workbench-images:code-server-ubi9-py39_2023b_latest
CUDA VSCode image on UBI9 with Python 3.9 quay.io/opendatahub-contrib/workbench-images:cuda-code-server-ubi9-py39_2023b_latest

RStudio 2022.12.0 + R 4.2.2

Image description Pull address
RStudio image on Centos Stream 9 with Python 3.9 quay.io/opendatahub-contrib/workbench-images:rstudio-c9s-py39_2023b_latest
CUDA RStudio image on Centos Stream 9 with Python 3.9 quay.io/opendatahub-contrib/workbench-images:cuda-rstudio-c9s-py39_2023b_latest

Specialty images

Image description Pull address
Spark/PySpark 3.3.1 with Hadoop 3.3.4 based on jupyter-datascience-ubi9-py39 quay.io/opendatahub-contrib/workbench-images:jupyter-spark-ubi9-py39_2023b_latest
OptaPy based on jupyter-datascience-ubi9-py39 quay.io/opendatahub-contrib/workbench-images:jupyter-optapy-ubi9-py39_2023b_latest
TrustyAI based on jupyter-datascience-ubi9-py39 quay.io/opendatahub-contrib/workbench-images:jupyter-trustyai-ubi9-py39_2023b_latest
Monai JupyterLab Notebook image on Centos Stream 9 with Python 3.9 quay.io/opendatahub-contrib/workbench-images:jupyter-monai-c9s-py39_2023b_latest

Runtimes

Image description Pull address
UBI9 based runtimes:
Minimal runtime on UBI9 with Python 3.9 quay.io/opendatahub-contrib/runtime-images:minimal-ubi9-py39_2023b_latest
CUDA Minimal runtime on UBI9 with Python 3.9 quay.io/opendatahub-contrib/runtime-images:minimal-cuda-ubi9-py39_2023b_latest
Datascience runtime on UBI9 with Python 3.9 quay.io/opendatahub-contrib/runtime-images:datascience-ubi9-py39_2023b_latest
CUDA Datascience runtime on UBI9 with Python 3.9 quay.io/opendatahub-contrib/runtime-images:datascience-cuda-ubi9-py39_2023b_latest
Spark runtime on UBI9 with Python 3.9 quay.io/opendatahub-contrib/runtime-images:spark-ubi9-py39_2023b_latest
CUDA Spark runtime on UBI9 with Python 3.9 quay.io/opendatahub-contrib/runtime-images:spark-cuda-ubi9-py39_2023b_latest
Optapy runtime on UBI9 with Python 3.9 quay.io/opendatahub-contrib/runtime-images:optapy-ubi9-py39_2023b_latest
Centos Stream 9 based runtimes:
Minimal runtime on C9S with Python 3.9 quay.io/opendatahub-contrib/runtime-images:minimal-c9s-py39_2023b_latest
CUDA Minimal runtime on C9S with Python 3.9 quay.io/opendatahub-contrib/runtime-images:minimal-cuda-c9s-py39_2023b_latest
Datascience runtime on C9S with Python 3.9 quay.io/opendatahub-contrib/runtime-images:datascience-c9s-py39_2023b_latest
CUDA Datascience runtime on C9S with Python 3.9 quay.io/opendatahub-contrib/runtime-images:datascience-cuda-c9s-py39_2023b_latest
Monai runtime on C9S with Python 3.9 quay.io/opendatahub-contrib/runtime-images:monai-c9s-py39_2023b_latest
R runtime on C9S with Python 3.9 quay.io/opendatahub-contrib/runtime-images:r-c9s-py39_2023b_latest
CUDA + R runtime on C9S with Python 3.9 quay.io/opendatahub-contrib/runtime-images:r-cuda-c9s-py39_2023b_latest
UBI8 based runtimes:
Minimal runtime on UBI8 with Python 3.8 quay.io/opendatahub-contrib/runtime-images:minimal/py38-ubi8-py38_2023b_latest
CUDA Minimal runtime on UBI8 with Python 3.8 quay.io/opendatahub-contrib/runtime-images:minimal/py38-cuda-ubi8-py38_2023b_latest
Datascience runtime on UBI8 with Python 3.8 quay.io/opendatahub-contrib/runtime-images:datascience/py38-ubi8-py38_2023b_latest
CUDA Datascience runtime on UBI8 with Python 3.8 quay.io/opendatahub-contrib/runtime-images:datascience/py38-cuda-ubi8-py38_2023b_latest

Base images

Different flavors are available, based on UBI8, UBI9 or Centos Stream 9, with different versions of Python:

source

CUDA base images

CUDA-enabled versions of the Base images:

  • CUDA version: 11.8.0
  • CuDNN version: 8.6.0

NOTE: the images listed here are the Runtime+CuDNN versions. The intermediate images from the CUDA build chain are not listed here, but available in the repo, as well as development images that may be needed to create further images when building packages from source. See the tree graphs below for image names.

source

Jupyter Images

All these images use JupyterLab as the main environment, which is started by default and listens on port 8888.

Minimal Notebook images

Minimal JupyterLab Notebook image with additional basic packages. The list and version of included packages is available in the README file in the corresponding folder.

Images derive from the Base image, with different flavors:

source

Datascience Notebook images

JupyterLab Notebook images with standard datascience packages, in different flavors (see above). The list and version of included packages is available in the README file in the corresponding folder.

source

Datascience + VSCode (Code-Server) Notebook image

Notebook based on the UBI9 Data Science notebook, adding VSCode (Code-Server).

source

Datascience + RStudio Notebook image

Notebook based on the Centos Stream9 Data Science notebook, adding an R Kernel for Jupyter and RStudio.

source

Spark image

Notebook based on the UBI9 Data Science notebook, adding Spark/PySpark capabilities.

source

Optapy image

Notebook based on the UBI9 Data Science notebook, adding OptaPy 8.31.1b0, the Python version of OptaPlanner.

source

CUDA-enabled Jupyter Images

CUDA Minimal Notebook images

CUDA-enabled Minimal JupyterLab Notebook image with additional basic packages. The list and version of included packages is available in the README file in the corresponding folder.

Images derive from the Base image, with different flavors:

source

CUDA Datascience Notebook images

CUDA-enabled JupyterLab Notebook images with standard datascience packages, in different flavors (see above). The list and version of included packages is available in the README file in the corresponding folder.

source

CUDA Datascience + VSCode (Code-Server) Notebook image

CUDA-enabled notebook based on the UBI9 Data Science notebook, adding VSCode (Code-Server).

source

CUDA Datascience + RStudio Notebook image

CUDA-enabled notebook based on the Centos Stream9 Data Science notebook, adding an R Kernel for Jupyter and RStudio.

source

CUDA Monai Notebook image

CUDA-enabled JupyterLab DataScience Notebook image with the addition of Monai 1.1.0 and OpenCV.

source

VSCode (Code-Server)

NOTE: The following images are Standalone VSCode (Code-Server) images, without the Jupyter environment. They are fully compatible with ODH and RHODS.

source

RStudio

NOTE: The following images are Standalone RStudio images, without the Jupyter environment. They are fully compatible with ODH and RHODS.

source

Development

Building images

All image folders contain a Makefile, as well as a README file with examples on how to use the Makefile.

The Makefile contains targets to build and to validate the image(s). The validation will check for the different binaries required depending on the image type, run a notebook in the image to validate the Python stack, and launch and try to access the different servers (JupyterLab, VSCode, RStudio,...). So even if the image does not do exactly what you want (open an issue!), it should always start properly.

Naming convention

Images have all the same name to fit in a single repo: workbench-images. The TAG is used to differentiate them with this scheme: {flavor}_{release}_{date}:

  • flavor is the type of the image, including its main purpose (e.g. "jupyterlab-datascience"), the base container image (e.g. "ubi9") and the Python version (e.g. "py39" for Python 3.9).
  • release is the release number, which follows a YearIndex pattern, e.g. 2023b for the first release of 2023, then 2023b,...
  • date is the date of the build of a given release. Images are built and updated on a weekly basis to apply patches and bug fixes, following semantic versioning. This means that for a given release number, major and minor versions of packages won't change (only z in version x.y.z).

Repository organization

This repository is applying the Github Flow strategy (new features or bug fixes branching from main) with a Git Flow twist: as we want to support different releases simultaneously and update them regularly (security patches or bug fixes), each new release (20XXy) will have its own support branch created and maintained, principally with weekly updates of the Pipfile.lock files to rebuild the images.

So the flow is:

  • Develop or fix on branches from the main branch.
  • Once a new release is ready to get out, create a branch with this release name.
  • Fixes or improvements that concern multiple branches (like this README) can be applied to main then cherry picked and applied to other relevant branches.
  • Images are built from their respective branch using the naming scheme described above.

Build chain

Notes:

  • All the images support S2I for Python to be easily extended.
  • UBI8 + Python 3.8 tree is mostly there for reference. There should not be further images based on it. Official ODH/RHODS images based on UBI8 and Python 3.8 are available here.
  • New development and images in this repo will be on UBI9 + Python 3.9 base, or CentOS Stream 9 where UBI9 is not possible due to lack of OS packages (for RStudio for example).

UBI9 Python 3.9 Tree

graph TB
    subgraph Main Tree
    %% Base
    ubi9-py39(UBI9 Python 3.9)
    base-ubi9-py39("Base<br/>(base-ubi9-py39)")
        %% links    
    ubi9-py39-->base-ubi9-py39
   
    %% Standard images
    minimal-ubi9-py39("Minimal Notebook<br/>(jupyter-minimal-ubi9-py39)")
    datascience-ubi9-py39("DataScience Notebook<br/>(jupyter-datascience-ubi9-py39)")
    datascience-code-server-ubi9-py39("DataScience + VSCode Notebook<br/>(jupyter-datascience-code-server-ubi9-py39)")
    spark-ubi9-py39("Spark Notebook<br/>(jupyter-spark-ubi9-py39)")
    optapy-ubi9-py39("OptaPy Notebook<br/>(jupyter-optapy-ubi9-py39)")
    code-server-ubi9-py39("VSCode<br/>(code-server-ubi9-py39)")
        %% links
    base-ubi9-py39-->minimal-ubi9-py39
    base-ubi9-py39-->code-server-ubi9-py39
    minimal-ubi9-py39-->datascience-ubi9-py39
    datascience-ubi9-py39-->datascience-code-server-ubi9-py39
    datascience-ubi9-py39-->spark-ubi9-py39
    datascience-ubi9-py39-->optapy-ubi9-py39
    
    %% CUDA images
    cuda-base-ubi9-py39("CUDA Base<br/>(cuda-base-ubi9-py39)")
    cuda-minimal-ubi9-py39("CUDA Minimal Notebook<br/>(cuda-jupyter-minimal-ubi9-py39)")
    cuda-datascience-ubi9-py39("CUDA DataScience Notebook<br/>(cuda-jupyter-datascience-ubi9-py39)")
    cuda-datascience-code-server-ubi9-py39("CUDA DataScience + VSCode Notebook<br/>(cuda-jupyter-datascience-code-server-ubi9-py39)")
    cuda-spark-ubi9-py39("CUDA Spark Notebook<br/>(cuda-jupyter-spark-ubi9-py39)")
    cuda-code-server-ubi9-py39("CUDA VSCode<br/>(cuda-code-server-ubi9-py39)")
        %% links
    base-ubi9-py39-.->|Through the CUDA build chain|cuda-base-ubi9-py39
    cuda-base-ubi9-py39-->cuda-minimal-ubi9-py39
    cuda-base-ubi9-py39-->cuda-code-server-ubi9-py39
    cuda-minimal-ubi9-py39-->cuda-datascience-ubi9-py39
    cuda-datascience-ubi9-py39-->cuda-datascience-code-server-ubi9-py39
    cuda-datascience-ubi9-py39-->cuda-spark-ubi9-py39
    end

    %% Links between the two graphs
    base-ubi9-py39-->cuda-ubi9-py39
    cuda-ubi9-py39-runtime-cudnn-->|Same image|cuda-base-ubi9-py39

    %% CUDA build chain
    subgraph CUDA Build Chain
    cuda-ubi9-py39("CUDA Base<br/>(cuda-ubi9-py39)")
    cuda-ubi9-py39-runtime("CUDA Runtime<br/>(cuda-ubi9-py39-runtime)")
    cuda-ubi9-py39-runtime-cudnn("CUDA Runtime cuDNN<br/>(cuda-ubi9-py39-runtime-cudnn)")
    cuda-ubi9-py39-devel("CUDA Devel<br/>(cuda-ubi9-py39-devel)")
    cuda-ubi9-py39-devel-cudnn("CUDA Devel cuDNN<br/>(cuda-ubi9-py39-devel-cudnn)")
        %% links
    cuda-ubi9-py39-->cuda-ubi9-py39-runtime
    cuda-ubi9-py39-runtime-->cuda-ubi9-py39-runtime-cudnn
    cuda-ubi9-py39-runtime-->cuda-ubi9-py39-devel
    cuda-ubi9-py39-devel-->cuda-ubi9-py39-devel-cudnn   
    end
Loading

CentOS Stream 9 Python 3.9 Tree

graph TB
    subgraph Main Tree
    %% base
    c9s-py39(Centos Stream 9 Python 3.9)
    base-c9s-py39("Base<br/>(base-c9s-py39)")
        %% links
    c9s-py39-->base-c9s-py39

    %% Standard images
    minimal-c9s-py39("Minimal Notebook<br/>(jupyter-minimal-c9s-py39)")
    datascience-c9s-py39("DataScience Notebook<br/>(jupyter-datascience-c9s-py39)")
    datascience-rstudio-c9s-py39("DataScience + RStudio Notebook<br/>(jupyter-datascience-rstudio-c9s-py39)")
    monai-c9s-py39("Monai Notebook<br/>(jupyter-monai-c9s-py39)")
    rstudio-c9s-py39("RStudio<br/>(rstudio-c9s-py39)")
        %% links
    base-c9s-py39-->minimal-c9s-py39
    base-c9s-py39-->rstudio-c9s-py39
    minimal-c9s-py39-->datascience-c9s-py39
    datascience-c9s-py39-->datascience-rstudio-c9s-py39
    datascience-c9s-py39-->monai-c9s-py39
    
    %% CUDA images
    cuda-base-c9s-py39("CUDA Base<br/>(cuda-base-c9s-py39)")
    cuda-minimal-c9s-py39("CUDA Minimal Notebook<br/>(cuda-jupyter-minimal-c9s-py39)")
    datascience-cuda-c9s-py39("CUDA DataScience Notebook<br/>(cuda-jupyter-datascience-c9s-py39)")
    cuda-jupyter-datascience-rstudio-c9s-py39("CUDA DataScience + RStudio Notebook<br/>(cuda-jupyter-datascience-rstudio-c9s-py39)")
    cuda-rstudio-c9s-py39("CUDA RStudio<br/>(cuda-rstudio-c9s-py39)")
        %% links
    base-c9s-py39-.->|Through the CUDA build chain|cuda-base-c9s-py39
    cuda-base-c9s-py39-->cuda-minimal-c9s-py39
    cuda-minimal-c9s-py39-->datascience-cuda-c9s-py39
    datascience-cuda-c9s-py39-->cuda-jupyter-datascience-rstudio-c9s-py39
    cuda-base-c9s-py39-->cuda-rstudio-c9s-py39
    end

    %% Links between the two graphs
    base-c9s-py39-->cuda-c9s-py39
    cuda-c9s-py39-runtime-cudnn-->|Same image|cuda-base-c9s-py39

    %% CUDA build chain
    subgraph CUDA Build Chain
    cuda-c9s-py39("CUDA Base<br/>(cuda-c9s-py39)")
    cuda-c9s-py39-runtime("CUDA Runtime<br/>(cuda-c9s-py39-runtime)")
    cuda-c9s-py39-runtime-cudnn("CUDA Runtime cuDNN<br/>(cuda-c9s-py39-runtime-cudnn)")
    cuda-c9s-py39-devel("CUDA Devel<br/>(cuda-c9s-py39-devel)")
    cuda-c9s-py39-devel-cudnn("CUDA Devel cuDNN<br/>(cuda-c9s-py39-devel-cudnn)")
        %% links
    cuda-c9s-py39-->cuda-c9s-py39-runtime
    cuda-c9s-py39-runtime-->cuda-c9s-py39-runtime-cudnn
    cuda-c9s-py39-runtime-->cuda-c9s-py39-devel
    cuda-c9s-py39-devel-->cuda-c9s-py39-devel-cudnn
    end
Loading

UBI8 Python 3.8 Tree

graph TB
    subgraph Main Tree
    %% Base
    ubi8-py38(UBI8 Python 3.8)
    base-ubi8-py38("Base<br/>(base-ubi8-py38)")
        %% links    
    ubi8-py38-->base-ubi8-py38
   
    %% Standard images
    minimal-ubi8-py38("Minimal Notebook<br/>(jupyter-minimal-ubi8-py38)")
    datascience-ubi8-py38("DataScience Notebook<br/>(jupyter-datascience-ubi8-py38)")
        %% links
    base-ubi8-py38-->minimal-ubi8-py38
    minimal-ubi8-py38-->datascience-ubi8-py38
    
    %% CUDA images
    cuda-base-ubi8-py38("CUDA Base<br/>(cuda-base-ubi8-py38)")
    cuda-minimal-ubi8-py38("CUDA Minimal Notebook<br/>(cuda-jupyter-minimal-ubi8-py38)")
    cuda-datascience-ubi8-py38("CUDA DataScience Notebook<br/>(cuda-jupyter-datascience-ubi8-py38)")
        %% links
    base-ubi8-py38-.->|Through the CUDA build chain|cuda-base-ubi8-py38
    cuda-base-ubi8-py38-->cuda-minimal-ubi8-py38
    cuda-minimal-ubi8-py38-->cuda-datascience-ubi8-py38
    end

    %% Links between the two graphs
    base-ubi8-py38-->cuda-ubi8-py38
    cuda-ubi8-py38-runtime-cudnn-->|Same image|cuda-base-ubi8-py38

    %% CUDA build chain
    subgraph CUDA Build Chain
    cuda-ubi8-py38("CUDA Base<br/>(cuda-ubi8-py38)")
    cuda-ubi8-py38-runtime("CUDA Runtime<br/>(cuda-ubi8-py38-runtime)")
    cuda-ubi8-py38-runtime-cudnn("CUDA Runtime cuDNN<br/>(cuda-ubi8-py38-runtime-cudnn)")
    cuda-ubi8-py38-devel("CUDA Devel<br/>(cuda-ubi8-py38-devel)")
    cuda-ubi8-py38-devel-cudnn("CUDA Devel cuDNN<br/>(cuda-ubi8-py38-devel-cudnn)")
        %% links
    cuda-ubi8-py38-->cuda-ubi8-py38-runtime
    cuda-ubi8-py38-runtime-->cuda-ubi8-py38-runtime-cudnn
    cuda-ubi8-py38-runtime-->cuda-ubi8-py38-devel
    cuda-ubi8-py38-devel-->cuda-ubi8-py38-devel-cudnn   
    end
Loading

About

Various custom Jupyter Notebooks

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Makefile 43.9%
  • Dockerfile 29.5%
  • Shell 11.2%
  • Python 7.0%
  • TypeScript 5.2%
  • Jupyter Notebook 2.8%
  • Other 0.4%