Skip to content

dbx-container

Build Documentation Docs with MkDocs Poetry linting: ruff pre-commit Checked with pyright security: bandit Semantic Versions Copier License

Create databricks runtime containers, vibe coded.

Docker images are losely based on the container definitions and runtime information.

Features

  • Generate Dockerfiles for all Databricks runtime versions
  • Support for multiple image types: minimal, standard, python (all as GPU variants where applicable)
  • LTS runtime support with ML variants
  • Automated CI/CD pipeline for building and publishing images
  • Multiple OS and Python version variations
  • Built-in runtime metadata and version tracking

Installation

With pip:

python -m pip install dbx-container

With poetry:

poetry add dbx-container

How to use it

Generate Dockerfiles

Generate Dockerfiles for all Databricks runtimes:

poetry run dbx-container build

Generate for a specific runtime:

poetry run dbx-container build --runtime-version "15.4 LTS"

Generate for a specific image type:

poetry run dbx-container build --image-type gpu

List Available Runtimes

View all supported Databricks runtime versions:

poetry run dbx-container list

Build Docker Images

Build all LTS images locally:

./scripts/build_images.sh

Build and push to a registry:

./scripts/build_images.sh --push --registry ghcr.io

Use Pre-built Images

Pull from GitHub Container Registry:

docker pull ghcr.io/twsl/dbx-runtime:python-17.3-lts-ubuntu2404-py312

Available Image Types

  • minimal - Base Ubuntu with Java (non-runtime-specific)
  • minimal-gpu - Base GPU image with CUDA and Java (non-runtime-specific)
  • standard - Standard with SSH server and FUSE support (non-runtime-specific)
  • standard-gpu - GPU standard with SSH and FUSE (non-runtime-specific)
  • python - Python runtime with virtualenv (runtime-specific)
  • python-gpu - GPU Python with CUDA support (runtime-specific)
  • gpu - Standalone GPU-enabled container (non-runtime-specific)

Each LTS runtime includes:

  • Base variant (standard runtime)
  • ML variant (machine learning runtime)
  • Multiple OS versions (Ubuntu 22.04, 24.04)
  • Appropriate Python versions (3.8-3.12)

CI/CD Pipeline

The project includes a GitHub Actions workflow that:

  1. Automatically generates Dockerfiles for all LTS runtimes
  2. Builds images in parallel using matrix strategy
  3. Pushes to GitHub Container Registry on main branch
  4. Supports manual triggering with filters

See Docker Build Guide for detailed documentation.

Docs

poetry run mkdocs build -f ./docs/mkdocs.yml -d ./_build/

Update template

copier update --trust -A --vcs-ref=HEAD

Credits

This project was generated with 🚀 A generic python project template.