Skip to content

Python Containers with uv/uvx: High-Performance Images, Minimal Memory, Minimal Start Time

Python is the worst language to containerize well and the most common language to containerize badly. The defaults are user-hostile: python:3.12 is 1 GB before you write a line of code; pip install is single-threaded and re-resolves the world; requirements.txt is unhashed; native wheels disagree with Alpine's musl; import pandas parses ~2,000 files before main() runs. Every layer of the stack — interpreter, packager, image, runtime — has a default that prioritizes "works for everybody" over "small, fast, secure."

This chapter is about flipping every one of those defaults. The thesis: with uv as the package manager, distroless or python:3.12-slim as the base, multi-stage builds, AOT-compiled .pyc, and tuned interpreter flags, a Python service can be packaged into a 50–100 MB image that cold-starts in 200–400 ms and uses 30–40% less RSS than the naive equivalent. Every dial along the way matters; most teams turn none of them.

If you read chapter 39 (Dockerfile best practices), this chapter is the language-specific deep dive that ties it to Python's particular pathologies. It assumes you're already comfortable with multi-stage builds, BuildKit cache mounts, and distroless bases — we'll go past those, into Python-specific territory. Sections 21–28 cover the advanced, surprising parts: what os.cpu_count() and the memory APIs see inside a cgroup, fork/COW and the GC, PID 1 signals, bytecode invalidation modes, Poetry vs uv, and build-cache deep cuts. For the machine underneath each dial, see the python-mastery/ track (§29 maps the two).


Table of Contents

  1. Why Python Defaults Are Hostile to Containers
  2. uv and uvx: What They Are and Why They Belong in Your Dockerfile
  3. The uv Mental Model: Lockfile, Resolver, Cache
  4. Base Image Choice for Python: A Reality Check
  5. The Canonical Multi-Stage uv Dockerfile
  6. Cache Mounts for uv: The Real Win
  7. uv sync vs uv pip install: Project Mode vs Pip Compat
  8. Bytecode Compilation: --compile-bytecode and Why It Matters
  9. Image Size: Where the Bytes Actually Go in Python Images
  10. Distroless Python: The Production Default
  11. uvx Inside the Build: Tools Without Polluting the Image
  12. Native Dependencies: psycopg, cryptography, numpy, lxml
  13. Memory: Interpreter Tuning, Allocator Tuning, Per-Worker Sizing
  14. Cold Start: The Import-Time Tax and How to Pay Less
  15. ASGI/WSGI Server Choice: uvicorn, gunicorn, granian, hypercorn
  16. __pycache__ Strategies and Read-Only Filesystems
  17. Reproducibility with uv lock
  18. CI/CD Patterns for uv Builds
  19. The Gold-Standard Dockerfile, Fully Annotated
  20. Measuring: How to Tell If You Actually Improved Anything
  21. CPU: What Python Thinks It Has vs What the cgroup Gives It
  22. Memory: Limits Python Can't See, OOMKills It Can't Catch
  23. Fork, Copy-on-Write, and the Garbage Collector
  24. PID 1, Signals, and Graceful Shutdown
  25. Bytecode Deep Cuts: Optimization Levels, Invalidation Modes, Version Tags
  26. Poetry vs uv (vs pip) in Containers
  27. Build Cache Deep Cuts
  28. The Surprise Catalog
  29. Cross-References: python-mastery/
  30. TL;DR

1. Why Python Defaults Are Hostile to Containers

Run through the data:

  • python:3.12 (Debian-based, full image): ~380 MB compressed, ~1.0 GB uncompressed.
  • python:3.12-slim: ~45 MB compressed, ~125 MB uncompressed. Drops dev headers, docs, locales.
  • python:3.12-alpine: ~20 MB compressed, ~55 MB uncompressed. musl libc.
  • gcr.io/distroless/python3-debian12: ~25 MB compressed, ~70 MB uncompressed. No shell, no package manager.

Then your dependencies pile on. pip install fastapi sqlalchemy psycopg2-binary adds another ~80 MB (FastAPI alone is small; psycopg2-binary ships a libpq inside the wheel; SQLAlchemy ships ~12 MB). Add pandas and you're +200 MB. Add tensorflow and you're at +2 GB. Add torch with CUDA and you're at +4 GB.

The problems compound:

  • Slow pip install. pip's backtracking resolver is correct but single-threaded, and downloads and installs serially. On a 50-dep project, a clean resolve can take 20–60 seconds.
  • No content-addressed cache by default. pip's wheel cache is per-user, not content-addressed; reinstalling the same package may or may not hit the cache.
  • __pycache__ written at first import. Cold start pays the .py → .pyc compilation tax every time, unless you precompile.
  • Import time dominated by module loading. import pandas executes several hundred modules (numpy, dateutil, pytz, its own submodules) before returning control, and parses every one of them too if there is no usable .pyc.
  • No native standard for lockfiles. pip freeze, requirements.txt, Pipfile.lock, poetry.lock, pdm.lock, pip-tools — six conventions, none built in.
  • musl vs glibc wheels split. Many wheels are manylinux2014 only; on Alpine, pip falls back to source builds. Now you need gcc, python3-dev, library headers — image triples in size.
  • Two-step builds without tooling support. Out of the box, building a wheel-only image (no compilers in production) requires hand-rolled stages.

uv and a tight Dockerfile fix every one of these.


2. uv and uvx: What They Are and Why They Belong in Your Dockerfile

uv is a Python package manager and resolver, written in Rust by Astral (the makers of Ruff). In one binary, it replaces:

  • pip (installing).
  • pip-tools (locking).
  • virtualenv / venv (env creation).
  • pipx (running Python CLIs in isolated envs — that's uvx).
  • Parts of poetry / pdm (project management with pyproject.toml + uv.lock).
  • pyenv (managing multiple Python versions).

The performance claims are not marketing — uv is 10–100× faster than pip on real workloads. On a fresh resolve of FastAPI + SQLAlchemy + Pydantic + uvicorn + ~20 deps, pip takes ~25 seconds; uv takes ~0.8 seconds. On warm cache, uv reuses already-extracted wheels via hardlinks and uv sync finishes a 50-dep install in well under a second.

Why this matters in a container:

  • CI builds drop 30–90 seconds. At 50 builds/day, that's a meaningful CI cost.
  • Cache mounts get utilized better. uv has a sane, content-addressed global cache (typically /root/.cache/uv) that BuildKit cache mounts handle cleanly.
  • Reproducibility is built-in. uv.lock is hashed, deterministic, and resolved with a PubGrub resolver; hashes are verified on install.
  • Bytecode compilation is one flag. uv sync --compile-bytecode or uv pip install --compile-bytecode precompiles .pyc at install time.
  • No interpreter required to bootstrap uv. uv is a static Rust binary; you don't need a Python in the image to install Python. (uv python install 3.12 will fetch a CPython if you don't have one.)

uvx is the run-a-Python-CLI-without-installing-it command:

uvx ruff check .
uvx black --check .
uvx pip-audit

uvx tools run in an isolated environment that does not contain your project or its dependencies. That's right for linters and formatters, which only read source, and wrong for anything that imports your code: uvx pytest or uvx mypy will fail on import yourapp. Run those from the project environment with uv run pytest (see §11).

In a Dockerfile build stage, uvx lets you run code-formatters, linters, test-runners, and migration tools without polluting the image — the tool runs in an ephemeral venv that lives in uv's cache, never in the final image. Compare to the bad pattern of pip install pytest && pytest && pip uninstall pytest (which leaves layer cruft regardless of the uninstall).


3. The uv Mental Model: Lockfile, Resolver, Cache

Three concepts:

  • pyproject.toml: declares dependencies, Python version constraints, and optional dependency groups.
  • uv.lock: the resolved, hashed lockfile. Committed to git. Cross-platform: contains entries for all platforms you care about.
  • Cache (~/.cache/uv or $UV_CACHE_DIR): a content-addressed store of downloaded wheels, extracted archives, built source distributions, and resolved version info. Reused across projects.

Workflow:

uv init                          # bootstrap pyproject.toml + .venv
uv add fastapi sqlalchemy        # add deps; updates pyproject.toml + uv.lock
uv sync                          # install everything in uv.lock into the venv
uv lock                          # re-resolve and write uv.lock without installing
uv lock --upgrade-package fastapi  # bump a single package
uv run python -m app             # run inside the venv without `source`

For Docker, the key insight: uv sync is fully deterministic given pyproject.toml + uv.lock + a Python version. Two identical uv sync invocations on the same lockfile produce identical site-packages directories, byte for byte (modulo .pyc mtimes, which we'll fix in §8 and §17).

uv and --locked: in CI, always use uv sync --locked (or --frozen). --locked errors if uv.lock would have to change to satisfy pyproject.toml; --frozen skips the freshness check entirely and installs the lockfile as-is. Both install exactly what the lockfile says; only --locked also proves the lockfile still matches pyproject.toml, so prefer it in CI.

RUN uv sync --locked --no-dev --no-install-project

Three flags worth knowing:

  • --no-dev: skip the dev dependency group. Production image doesn't need pytest, mypy, ruff.
  • --no-install-project: don't install the project itself (yet). We'll copy source and install in a later step to get cache layering right.
  • --locked: fail if lockfile is stale.

4. Base Image Choice for Python: A Reality Check

The honest comparison for production:

Base Size Wheels Shell Verdict
python:3.12 ~1 GB Both Yes Never for runtime. Sometimes for build stage.
python:3.12-slim-bookworm ~45 MB glibc Yes Default for build stage. Reasonable for runtime if you need a shell.
python:3.12-alpine ~20 MB musl (often missing) Yes Avoid unless you've audited every wheel.
gcr.io/distroless/python3-debian12 ~25 MB glibc No Production runtime default, but it is Debian's Python 3.11: the builder must match (§25).
gcr.io/distroless/cc-debian12 + uv-managed Python ~40 MB glibc No Distroless with the interpreter version you choose. Used in §19.
cgr.dev/chainguard/python:latest ~30 MB glibc No Chainguard's distroless equivalent. Aggressively CVE-tracked.
ghcr.io/astral-sh/uv:python3.12-bookworm-slim ~85 MB glibc Yes Bundles uv + Python; convenient for build stages.

The Alpine warnings are not theoretical. Specific failures you'll encounter:

  • Wheel coverage lags on musl. Recent psycopg2-binary, numpy and cryptography do publish musllinux wheels, but older pins, smaller packages, and new Python versions often don't. Any gap falls back to a source build → gcc, headers, and minutes per build.
  • numpy, pandas, scipy, scikit-learn ship manylinux wheels first; musllinux wheels are newer and sometimes incomplete for older versions.
  • cryptography has musl wheels but requires rustc to build older versions.
  • DNS behaves differently (single-label, parallel queries — see ch 39 §3).
  • Threading stack sizes are smaller; deep recursion in Python (e.g., parsing nested JSON) can crash.

The two-line rule: build in python:3.12-slim-bookworm (or the astral-sh/uv image); ship in distroless :nonroot, with the same Python minor version in both stages. Everything else is a deviation that should be justified.

When not to use distroless:

  • Your app shells out to subprocesses (calls subprocess.run(['ffmpeg', ...])). Distroless has no ffmpeg, no shell. Either bundle the binary you need into the image (copy it from a separate stage) or use python:3.12-slim for runtime.
  • You need to kubectl exec for ad-hoc debugging. Use kubectl debug with an ephemeral debug container instead; it's the right tool.
  • Your app uses os.system or any shell=True subprocess call. Refactor that — but if you can't, you need a shell in the image.

5. The Canonical Multi-Stage uv Dockerfile

The starting point. We'll improve on this in later sections, but get this in your head as the baseline:

# syntax=docker/dockerfile:1.7

# ----- builder ----------------------------------------------------------
FROM ghcr.io/astral-sh/uv:python3.12-bookworm-slim AS builder

ENV UV_COMPILE_BYTECODE=1 \
    UV_LINK_MODE=copy \
    UV_PYTHON_DOWNLOADS=never \
    PYTHONDONTWRITEBYTECODE=1 \
    PYTHONUNBUFFERED=1

WORKDIR /app

# Install deps first (cache-friendly)
COPY pyproject.toml uv.lock ./
RUN --mount=type=cache,target=/root/.cache/uv \
    uv sync --locked --no-dev --no-install-project

# Now copy the project and install it as a regular (non-editable) package
COPY src ./src
COPY README.md ./
RUN --mount=type=cache,target=/root/.cache/uv \
    uv sync --locked --no-dev --no-editable

# ----- runtime ----------------------------------------------------------
FROM gcr.io/distroless/python3-debian12:nonroot

ENV PYTHONDONTWRITEBYTECODE=1 \
    PYTHONUNBUFFERED=1 \
    PATH="/app/.venv/bin:${PATH}"

WORKDIR /app

# Copy the built venv from the builder; the project lives inside it
COPY --from=builder --chown=nonroot:nonroot /app/.venv /app/.venv

USER nonroot
EXPOSE 8000
ENTRYPOINT ["python", "-m", "app.main"]

The project uses the standard src layout, with a real package name and a build backend so uv sync can install it:

pyproject.toml        # [project] name = "app" ... and:
                      # [build-system]
                      # requires = ["uv_build>=0.8,<0.12"]
                      # build-backend = "uv_build"
uv.lock
src/app/__init__.py
src/app/main.py       # app = FastAPI()
tests/test_main.py    # from app.main import app

Without a [build-system] table, uv treats the project as a virtual project and never installs it, so import app only works when the working directory happens to be on sys.path.

Version trap. distroless/python3-debian12 ships Debian's Python 3.11. With a 3.12 builder, the venv's python symlink points at /usr/local/bin/python3.12, which doesn't exist in the runtime image, and the cpython-312 .pycs would be ignored anyway. Either match the builder to the runtime (ghcr.io/astral-sh/uv:python3.11-bookworm-slim) or ship your own interpreter as §19 does. See §25.

What's happening, line by line:

  • # syntax=docker/dockerfile:1.7: pin BuildKit frontend.
  • FROM ghcr.io/astral-sh/uv:python3.12-bookworm-slim AS builder: starts from Astral's image which has both uv and Python 3.12 already, on Debian slim. Saves an apt install step.
  • UV_COMPILE_BYTECODE=1: tells uv to compile .py → .pyc at install time. (Equivalent to passing --compile-bytecode.)
  • UV_LINK_MODE=copy: forces uv to copy wheel contents into the venv rather than hardlink. Hardlinks are faster but impossible when the cache mount is on a different filesystem than the project — which it always is in BuildKit. Without this, uv tries, fails, and falls back to copying with a Failed to hardlink files warning on every build; setting it makes the intent explicit and the logs clean.
  • UV_PYTHON_DOWNLOADS=never: uv won't try to download a Python interpreter; it uses the one in the image. Saves bytes and avoids surprises.
  • PYTHONDONTWRITEBYTECODE=1: don't write .pyc at runtime (we already did it at build time).
  • PYTHONUNBUFFERED=1: flush stdout/stderr after every write. Critical for container logging: without this, your logs are buffered until the process exits or fills the buffer.
  • COPY pyproject.toml uv.lock before source: cache layer for dependencies.
  • --mount=type=cache,target=/root/.cache/uv: BuildKit cache mount on uv's global cache. Massive speedup across builds.
  • uv sync --locked --no-dev --no-install-project: install dependencies but not the project itself yet.
  • COPY src then second uv sync --no-editable: the project is built and installed into .venv/lib/python3.12/site-packages/app/ like any dependency, so UV_COMPILE_BYTECODE compiles it too, and the runtime stage needs only .venv. (The default, editable, would leave a .pth pointing at /app/src: the source would have to be copied separately and would never be precompiled; see §25.)
  • Final stage: distroless Python, copies the prebuilt .venv (which now contains the app). No uv, no pip, no shell in the runtime image.
  • USER nonroot: distroless ships a nonroot user at UID 65532. Use it.

Resulting image: ~75–90 MB compressed for a typical FastAPI/SQLAlchemy service. Compare to ~450–550 MB compressed (over 1 GB uncompressed) for a python:3.12 + pip install build of the same code.


6. Cache Mounts for uv: The Real Win

uv's cache directory is the secret sauce. By default, on Linux it's ~/.cache/uv (or $UV_CACHE_DIR). It contains:

  • Wheel cache: downloaded .whl files, content-addressed.
  • Extracted wheel cache: pre-extracted wheel contents (so install is just copy/hardlink).
  • Source distribution cache: .tar.gz sdists.
  • Built wheels: if uv had to build a wheel from an sdist (because no binary wheel was available), it caches the built wheel.
  • Resolver cache: cached metadata for already-queried packages.

A BuildKit cache mount keeps this directory across builds:

RUN --mount=type=cache,target=/root/.cache/uv \
    uv sync --locked --no-dev --no-install-project

Cache hit rates in practice:

  • Same project, same lockfile, different code change: ~100% hit. The uv sync line completes in <1 second.
  • Same project, dep version bump: ~95% hit (only the changed package is re-downloaded).
  • Different project, overlapping deps: significant hit (this is the global-cache magic — numpy==1.26.4 is in the cache regardless of which project pulled it).

Two important details:

1. Set UV_LINK_MODE=copy with cache mounts.

uv's default link mode is hardlink on Linux: when installing a wheel into a venv, it creates hardlinks from the cache to the venv. Hardlinks are fast and disk-efficient — but they require the source and destination to be on the same filesystem. In BuildKit, the cache mount is a separate filesystem from the build's working directory, so hardlinking fails with EXDEV and uv falls back to copying, with a warning.

Set UV_LINK_MODE=copy (or pass --link-mode=copy) to copy from the start: same result, no failed attempt, no warning. Copying is also what you want anyway, since the final image must contain real files, not links into a cache that won't exist at runtime. Net result: still much faster than pip.

2. Cache mount sharing across stages.

If you have multiple stages doing uv sync (e.g., one for dev deps to run tests, another for production deps), they share the cache mount. No duplicate downloads:

FROM ... AS test
RUN --mount=type=cache,target=/root/.cache/uv \
    uv sync --locked            # includes dev deps
RUN uv run pytest

FROM ... AS prod
RUN --mount=type=cache,target=/root/.cache/uv \
    uv sync --locked --no-dev   # production only

Both stages share /root/.cache/uv. The test stage downloads pytest; the prod stage doesn't install it, but numpy (used by both) was downloaded once.


7. uv sync vs uv pip install: Project Mode vs Pip Compat

uv has two main install commands:

  • uv sync: project mode. Reads pyproject.toml and uv.lock. Manages a single venv (typically .venv). The canonical command for project workflows.
  • uv pip install: pip-compatible mode. Reads requirements.txt. Installs into the current Python environment. Useful for legacy projects or when you don't want a pyproject.toml.

In a new project, use uv sync. In a legacy project with requirements.txt, you have two options:

Option A: migrate to pyproject.toml + uv.lock.

uv init --no-readme --no-pin-python
# Hand-edit pyproject.toml to add dependencies, or:
uv add -r requirements.txt     # handles comments, markers, and extras
uv lock

Option B: keep requirements.txt, use uv pip install.

COPY requirements.txt ./
RUN --mount=type=cache,target=/root/.cache/uv \
    uv pip install --system --no-deps -r requirements.txt

--system installs into the system Python (no venv). --no-deps is dangerous unless you trust your requirements.txt to be fully resolved (use uv pip compile requirements.in -o requirements.txt --generate-hashes to produce a hashed, fully-resolved file).

For new projects, always use uv sync with pyproject.toml + uv.lock. Lockfile-driven workflows are the future; requirements.txt is legacy.

Bonus tip — uv pip compile:

uv pip compile requirements.in -o requirements.txt --generate-hashes

This is the pip-tools replacement. Even if you keep requirements.txt, generating it from requirements.in with uv pip compile --generate-hashes gives you a hashed, fully-pinned, reproducible install file. The Dockerfile can then uv pip install --require-hashes -r requirements.txt for hash-verified installs.


8. Bytecode Compilation: --compile-bytecode and Why It Matters

When Python imports a module, it:

  1. Looks for a .pyc file (compiled bytecode) in __pycache__/.
  2. If found and the source's mtime/hash matches: load the .pyc.
  3. If not found or stale: parse the .py, compile to bytecode, write .pyc, then execute.

On a cold container with a read-only filesystem, step 3 fails to write but still does the parse-and-compile work — every time the process starts. For a service that imports a couple thousand modules at startup (Django + DRF + a few apps; FastAPI + SQLAlchemy + Alembic; anything ML-flavored), compilation can take 300–800 ms at every boot.

uv sync --compile-bytecode (or UV_COMPILE_BYTECODE=1) compiles .pyc at install time, in the build stage. The .pyc ships in the image. Cold start now skips compilation entirely:

# Without --compile-bytecode
container start (350ms)
+ import django (220ms parse + 80ms execute)
+ import drf (180ms parse + 50ms execute)
+ ... = ~700ms before first request

# With --compile-bytecode
container start (350ms)
+ import django (45ms load .pyc + 80ms execute)
+ import drf (30ms load .pyc + 50ms execute)
+ ... = ~250ms before first request

Real numbers from a FastAPI service: cold start drops from ~1.2 s to ~0.4 s with bytecode compilation. Just flip the flag. It costs some build time (seconds, parallelized) and image size (the .pyc files are roughly as large as the .py sources they sit next to). Both are paid once per build instead of on every pod start. Two traps can silently undo it: running with PYTHONOPTIMIZE, and an editable project install (§25).

A related trick — PYTHONDONTWRITEBYTECODE=1 at runtime — prevents Python from writing new .pyc files at runtime. Combined with read-only root filesystems (readOnlyRootFilesystem: true in K8s), this means:

  • Build stage compiles .pycs into the image.
  • Runtime has all the .pycs available.
  • Runtime can't write new .pycs (no need to, no permission to).
  • No write attempts to a read-only FS that would fail loudly.

This is the right configuration. Both flags. Always.


9. Image Size: Where the Bytes Actually Go in Python Images

Run dive on a typical Python service image. The breakdown:

Component Size Reducible?
Base OS (distroless or slim) 25–45 MB Choose smaller base
Python interpreter 25–30 MB Already minimal
site-packages (dependencies) 50–200 MB Yes (next sections)
Your code 0.5–5 MB Already minimal
__pycache__ 10–30 MB Keep for cold start (see note below on stripping .py)

The dependency layer is where the savings are. Specific patterns:

Strip tests and docs from wheels.

Many wheels ship their tests/ directory and docs. Examples: numpy ships a tests/ directory (~30 MB), scipy (~50 MB), pandas (~15 MB). These are useless in production. Strip them in the build stage:

RUN find /app/.venv -type d -name tests -prune -exec rm -rf {} + \
 && find /app/.venv -type d -name __pycache__ -prune \
        -path '*/tests/*' -exec rm -rf {} + \
 && find /app/.venv -type d -name "*.dist-info" -exec sh -c \
        'rm -rf "$1/RECORD" "$1/INSTALLER" "$1/REQUESTED"' _ {} \;

That find typically saves 30–100 MB on data-science-flavored images.

Strip the C extension .so debug info.

Many wheels ship native extensions (.so files) with debug symbols. strip them:

RUN find /app/.venv -name "*.so" -exec strip --strip-unneeded {} + 2>/dev/null || true

Saves 20–50 MB for ML-heavy images (many manylinux wheels are already stripped; measure). Two cautions: strip comes from binutils, which slim and uv images don't include, so with 2>/dev/null || true the step silently does nothing until you apt-get install binutils in the builder; and the || true also hides real failures, so check the size before and after once.

Sourceless images (shipping .pyc without .py). It's tempting to delete the .py files and keep __pycache__. That doesn't work: __pycache__/*.pyc files are only used when the matching .py exists. Sourceless distribution needs the legacy layout, python -m compileall -b (writes module.pyc next to the source), then deleting the .py. You lose readable tracebacks and save roughly the source size. Rarely worth it, and not an IP-protection mechanism (bytecode decompiles).

Remove __pycache__ for unused Python versions.

If wheels ship .pyc files for multiple Python versions in their __pycache__/ (look for *.cpython-39.pyc, *.cpython-310.pyc, *.cpython-311.pyc, etc.), strip those that don't match your runtime:

# Keep only 3.12 bytecode
RUN find /app/.venv -name "__pycache__" -type d -exec sh -c \
    'find "$1" -name "*.cpython-*.pyc" ! -name "*.cpython-312.pyc" -delete' _ {} \;

Not common (most wheels only ship .py, with .pyc compiled at install) but worth knowing.

Drop locale data.

locale/ directories under site-packages are huge for libraries with translations (Django, Babel, polib). If your app is English-only:

RUN find /app/.venv -path "*/locale/*" \( -name "*.mo" -o -name "*.po" \) -delete

This is aggressive — your error messages won't be localized — but it's a 10–40 MB save for some apps.

Beware of pip install --no-deps followed by manual deps.

A common "minimize" pattern is to install only what you import. It works, but: dependency graphs are deep, and you'll get confusing import errors at runtime instead of build time. Use --no-deps only when you have a fully-resolved hashed requirements file.


10. Distroless Python: The Production Default

gcr.io/distroless/python3-debian12 includes:

  • Debian's CPython interpreter: 3.11 on debian12 (3.13 on debian13), not the version you pick. If your builder uses a different minor version, the venv won't run on it (§25). The gold-standard Dockerfile in §19 avoids this by shipping a uv-managed interpreter on distroless/cc.
  • The CPython standard library.
  • glibc, ca-certificates, tzdata, /etc/passwd with a nonroot user.

It does NOT include:

  • pip, setuptools, wheel (none of them are needed at runtime).
  • A shell. No sh, no bash, no busybox.
  • Coreutils, curl, wget.
  • apt, dpkg.

For a Python service, this is fine — you're running python -m yourapp (or your venv's python). No shell needed.

Variant tags:

  • :latest — current version, root user.
  • :nonroot — current version, runs as UID 65532.
  • :debug — adds a busybox shell. Useful for ad-hoc debugging in CI; do not ship to production.
  • :debug-nonroot — both.

Use :nonroot in production. It maps to runAsUser: 65532 in your pod spec.

Healthchecks in distroless: there's no curl. On Kubernetes this mostly doesn't matter: httpGet, tcpSocket and grpc probes are executed by the kubelet, not inside the container, so they need nothing in the image and cost no in-container CPU. Prefer them:

readinessProbe:
  httpGet: { path: /healthz/ready, port: 8000 }
  periodSeconds: 5
livenessProbe:
  httpGet: { path: /healthz, port: 8000 }
  periodSeconds: 10

The curl problem is real for Docker HEALTHCHECK (Compose, ECS) and for exec probes. Two options there:

1. Use a Python-based healthcheck.

livenessProbe:
  exec:
    command: ["python", "-c", "import urllib.request,sys; urllib.request.urlopen('http://127.0.0.1:8000/healthz', timeout=2)"]
  periodSeconds: 10

Adds ~50–80 ms per probe (interpreter startup). Fine for liveness; might be slow for readiness if you probe aggressively.

2. Bundle a tiny health probe binary.

Compile a 5-line Go binary that does an HTTP GET and exits 0/1. Copy it into the image. Distroless still works, probe takes <5 ms.

COPY --from=build /healthcheck /healthcheck
livenessProbe:
  exec:
    command: ["/healthcheck", "http://127.0.0.1:8000/healthz"]

grpc_health_probe does this for gRPC (on Kubernetes 1.24+, the native grpc: probe replaces it).


11. uvx Inside the Build: Tools Without Polluting the Image

uvx (alias for uv tool run) executes a Python CLI in an ephemeral venv, similar to pipx run. Critically, the tool's installation does not affect the project's venv — it lives in uv's cache.

Use cases inside a Dockerfile:

# Run formatter / linter as a build-time check
RUN --mount=type=cache,target=/root/.cache/uv \
    uvx ruff check src/

# Pin tool version explicitly
RUN --mount=type=cache,target=/root/.cache/uv \
    uvx ruff@0.6.4 check src/

# Audit the locked dependencies for known CVEs
RUN --mount=type=cache,target=/root/.cache/uv \
    uv export --locked --no-dev --format requirements-txt > /tmp/req.txt \
 && uvx pip-audit -r /tmp/req.txt --disable-pip

What not to run with uvx: tests and type checkers. uvx pytest runs pytest in an environment without your project or its dependencies, so collection fails on the first import. uvx mypy can't see your dependencies' types. Those belong to the project environment (uv run, below).

None of these tools end up in the final image because:

  1. uvx installs into a separate venv (cache-mounted, ephemeral).
  2. The final stage copies only /app/.venv, not the uvx cache.

This is much cleaner than the legacy:

# Bad: pollutes the venv
RUN pip install ruff && ruff check src/ && pip uninstall -y ruff

The pip uninstall doesn't remove the disk bytes from the layer (image is a stack of immutable layers). uvx sidesteps this entirely.

uv run is the project equivalent.

For tools that are dev dependencies of your project (in the dev group of pyproject.toml):

RUN --mount=type=cache,target=/root/.cache/uv \
    uv sync --locked    # installs dev deps too
RUN uv run pytest

Do this in a separate test stage (§18) so dev dependencies never reach the runtime image. uvx is for tools that don't import your code; uv run in a test stage is for tools that do.


12. Native Dependencies: psycopg, cryptography, numpy, lxml

Python wheels often include C extensions. The wheel format includes prebuilt binaries for specific (Python version, platform, libc) tuples. If a wheel matching your environment exists, install is fast and stable. If not, pip/uv falls back to building from source, which needs compilers.

Common offenders and their fixes:

psycopg2-binary vs psycopg2 vs psycopg

  • psycopg2-binary ships libpq inside the wheel. Easy install, but a duplicated libpq per container (security hygiene problem — when libpq has a CVE, you wait for psycopg2-binary to release a new wheel).
  • psycopg2 (no -binary) needs system libpq at install time and runtime. Requires apt-get install libpq-dev in build stage and libpq5 in runtime stage. Smaller image, system-managed libpq.
  • psycopg (psycopg3) is the modern replacement; psycopg[binary] is the easy version. New code should use psycopg3.

For most projects: psycopg[binary] is fine. For regulated environments: psycopg linked against system libpq.

cryptography

Native wheels exist for glibc and musl on amd64 and arm64. Should "just work" with uv on modern bases.

If you see source builds: you've probably pinned an old version. cryptography>=41 has comprehensive wheels.

numpy, pandas, scipy

manylinux wheels are reliable on python:3.12-slim. On Alpine, you need musl wheels (sometimes available) or a source build (slow, needs OpenBLAS/LAPACK dev libraries). Don't put data-science workloads on Alpine.

For ML images, consider:

  • Use python:3.12-slim with prebuilt wheels.
  • Skip MKL unless you need it (MKL adds ~500 MB and rarely matters for inference).
  • For GPU: use NVIDIA's CUDA base images, install Python + uv on top.

lxml

Has glibc and musl wheels for current versions. Older Alpine + older lxml = source build needing libxml2-dev, libxslt-dev, gcc.

Generic pattern for unavoidable source builds

If you must build from source, use a multi-stage:

FROM ghcr.io/astral-sh/uv:python3.12-bookworm-slim AS builder

RUN apt-get update && apt-get install -y --no-install-recommends \
        gcc g++ libpq-dev libxml2-dev libxslt-dev \
    && rm -rf /var/lib/apt/lists/*

ENV UV_COMPILE_BYTECODE=1 \
    UV_LINK_MODE=copy

WORKDIR /app
COPY pyproject.toml uv.lock ./
RUN --mount=type=cache,target=/root/.cache/uv \
    uv sync --locked --no-dev --no-install-project
COPY src ./src
RUN --mount=type=cache,target=/root/.cache/uv \
    uv sync --locked --no-dev --no-editable

# Runtime: no compilers, only runtime libs.
# Same Debian release and same Python minor version as the builder.
FROM python:3.12-slim-bookworm
RUN apt-get update && apt-get install -y --no-install-recommends \
        libpq5 libxml2 libxslt1.1 \
    && rm -rf /var/lib/apt/lists/*
COPY --from=builder /app/.venv /app/.venv
WORKDIR /app
USER 65532
ENV PATH="/app/.venv/bin:${PATH}"
ENTRYPOINT ["python", "-m", "app.main"]

The runtime image gets the runtime library packages, not the -dev packages and compilers. Use ldd /app/.venv/lib/python3.12/site-packages/somepackage/_internals.cpython-312-x86_64-linux-gnu.so to discover which libs each native extension needs.

Why not hand-copy the .so files into distroless? It's a common recipe and it's fragile: libpq.so.5 alone pulls in Kerberos, LDAP, and SASL libraries; libxml2 pulls in ICU and liblzma; hard-coded /usr/lib/x86_64-linux-gnu/ paths break the arm64 build; and nothing tells you when a Debian security update changes the set. If you need distroless here, copy the full ldd closure with a script and test the image by importing every native module in CI (python -c "import psycopg2, lxml.etree").

PyTorch and other CUDA-sized wheels

The default torch wheel on Linux pulls in the CUDA runtime (several GB). For CPU-only inference, point uv at PyTorch's CPU index; images drop by 2–4 GB:

[tool.uv.sources]
torch = { index = "pytorch-cpu" }

[[tool.uv.index]]
name = "pytorch-cpu"
url = "https://download.pytorch.org/whl/cpu"
explicit = true

13. Memory: Interpreter Tuning, Allocator Tuning, Per-Worker Sizing

Python's memory characteristics in containers:

  • Resident set size (RSS) is what the kernel counts toward cgroup limits.
  • Python rarely returns memory to the OS, so RSS grows toward the high-water mark and stays there.
  • Objects ≤ 512 bytes come from pymalloc arenas; everything larger (and every C extension) goes to glibc malloc, whose per-thread arenas make multi-threaded apps consume more memory than single-threaded equivalents (§22).

Sizing: how much memory does a Python process need?

Rough order-of-magnitude:

  • Bare Python 3.12 interpreter: 10–15 MB RSS.
  • Plus Django + DRF + a few apps: 50–80 MB.
  • Plus FastAPI + SQLAlchemy + uvicorn: 60–90 MB.
  • Plus pandas and numpy imported: 120–180 MB.
  • Plus a 10 MB ORM cache warmed up: + actual data size.

Set Kubernetes requests.memory near the post-warmup RSS and limits.memory ~2× that to allow growth. Too-tight limits cause OOMKill on transient spikes (large request bodies, ORM queries returning lots of rows, periodic batch jobs).

PYTHONMALLOC and the allocator

Python has multiple memory allocators. The default pymalloc is fast for small allocations. For diagnostic builds, you can use PYTHONMALLOC=malloc to bypass pymalloc and route everything to libc malloc — useful for tracking down memory issues with tools that hook libc malloc (valgrind, address sanitizer). In production, leave it default.

MALLOC_ARENA_MAX for multi-threaded workloads

glibc's malloc allocates per-thread arenas. By default, it can use up to 8 × num_cpus arenas. On a 16-core machine, that's 128 arenas per process. Each arena reserves virtual address space and over time can hold significant unused memory.

For containerized Python workloads (especially uvicorn with multiple workers, or any process with many threads):

ENV MALLOC_ARENA_MAX=2

This caps glibc arenas at 2. RSS savings can be 20–40% on multi-threaded apps. Tradeoff: slightly more lock contention on malloc, rarely measurable for normal Python workloads (the GIL serializes most allocation anyway).

PYTHONHASHSEED and security

PYTHONHASHSEED=random (the default in Python 3) randomizes hash seeds per process. Important for security (prevents algorithmic complexity attacks against dicts).

PYTHONHASHSEED=0 disables this. Don't. The performance gain is nil; the security cost is real.

PYTHONUNBUFFERED=1 and logging

Without this, stdout is line-buffered when attached to a TTY and block-buffered when not (pipes, files, container log streams). In a container, stdout is piped to the container runtime → block-buffered → print() output appears late, out of order relative to stderr, and is lost entirely on SIGKILL/OOMKill. (stderr, where logging writes by default, has been line-buffered since 3.9, which is why logs often look fine while prints don't.)

Always set PYTHONUNBUFFERED=1. Costs nothing.

PYTHONOPTIMIZE

  • PYTHONOPTIMIZE=1: removes assert statements and sets __debug__ to False. Small startup speedup, smaller .pyc files.
  • PYTHONOPTIMIZE=2: also removes docstrings. ~5–10% memory savings on doc-heavy libraries, and it breaks anything that reads __doc__ at runtime: FastAPI's OpenAPI descriptions, Click/Typer help text, PLY grammars, some plugin systems.

If you rely on assertions in production code, don't use this. And if you set it, you must compile bytecode at the same level: uv --compile-bytecode produces level-0 .pycs only, so PYTHONOPTIMIZE=1 makes the interpreter ignore them and recompile from source on every start (§25). The gold-standard Dockerfile leaves it off.

# Only together with: python -m compileall -q -j 0 -o 1 /app/.venv/lib
ENV PYTHONOPTIMIZE=1

Forking and copy-on-write with gunicorn --preload

If you use gunicorn with --preload, the master process imports all your code once, then forks workers. The workers share memory via copy-on-write (COW) — until they touch a page, which then gets duplicated. For Django/Flask apps, this can save 30–60% RSS across all workers.

gunicorn --workers 4 --preload --worker-class uvicorn_worker.UvicornWorker app:application

The catch: any code that opens connections, file descriptors, or starts threads at import time will share them across workers, which is broken. Defer such initialization to a post_fork hook or worker startup.


14. Cold Start: The Import-Time Tax and How to Pay Less

Cold start is dominated by imports. Profile your imports:

python -X importtime -m yourapp 2> import.log      # or PYTHONPROFILEIMPORTTIME=1

The log shows hierarchical time spent in each import. Read it; you'll be surprised what's expensive.

Common offenders (Python 3.12 measurements on a modern Linux box):

  • import numpy: ~80 ms (imports linalg, fft, polynomial, ma, ctypeslib, etc.).
  • import pandas: ~250 ms (imports numpy + a hundred submodules).
  • import django: ~150 ms (imports settings, ORM, template engine).
  • import requests: ~50 ms (imports urllib3, charset_normalizer, idna, certifi).
  • import boto3: ~300 ms (loads service descriptors for ~300 AWS services).
  • import google.cloud.storage: ~400 ms (similar story).

Reduce imports at startup

Lazy import frequent vs rare paths. If only 1% of requests need boto3.client('s3'), defer the import:

def upload_to_s3(blob: bytes, key: str) -> None:
    import boto3
    s3 = boto3.client("s3")
    s3.put_object(...)

Now boto3 is imported once per process (on first call), not at startup. Cold start drops by 300 ms.

Use __getattr__ at the module level (PEP 562) for optional features.

# package/__init__.py
def __getattr__(name):
    if name == "HeavyClass":
        from .heavy_module import HeavyClass
        return HeavyClass
    raise AttributeError(name)

from package import HeavyClass works as expected, but import package doesn't pull in heavy_module.

Skip pkg_resources.

pkg_resources (from setuptools) is notoriously slow to import (~200 ms). Any library that does import pkg_resources at module load pays this. The modern replacement is importlib.metadata (stdlib, fast). Audit your deps.

Precompiled .pyc (revisited)

Covered in §8 — UV_COMPILE_BYTECODE=1 is a free 300–500 ms cold-start win on import-heavy apps. Always do it.

PYTHONDONTWRITEBYTECODE=1 and the read-only filesystem

Once you have precompiled .pycs in the image, you don't want runtime to write more. PYTHONDONTWRITEBYTECODE=1 + readOnlyRootFilesystem: true is the right combo. Bonus: prevents an attacker from writing executable bytecode into the container.

python -S (skip site)

Python's site.py runs at startup, scanning site-packages and processing .pth files. For ultra-lean services, python -S -m yourapp skips this. But site.py is also what puts the venv's site-packages on sys.path (via pyvenv.cfg; activation scripts only change PATH), so with -S none of your dependencies import until you set PYTHONPATH=/app/.venv/lib/python3.12/site-packages yourself, and .pth-based editable installs stop working. Saves ~10–30 ms. Only worth it for CLIs started thousands of times; for a long-running service it's noise.

Instrumentation and plugin discovery

Auto-instrumentation (opentelemetry-instrument, APM agents) and entry-point plugin discovery (importlib.metadata.entry_points(), pytest-style plugin scanning) import extra packages and scan every installed distribution's metadata at startup. Measure them with -X importtime like any other import; enable only the instrumentations you use.

"Warm up" the process before declaring it ready

If your app has expensive first-request paths (loading an ML model, opening DB connection pools, JIT compiling a regex), do them at startup, not at first request. Mark readiness only after warm-up completes. Otherwise the first request after pod start is slow, which gets routed to a probe-failing pod, which marks as not-ready, which causes oscillation under autoscaling.

from contextlib import asynccontextmanager

from fastapi import FastAPI, Response


@asynccontextmanager
async def lifespan(app: FastAPI):
    app.state.ready = False
    await db.connect()
    await load_model()
    app.state.ready = True
    yield
    await db.disconnect()


app = FastAPI(lifespan=lifespan)


@app.get("/healthz/ready")
def ready() -> Response:
    return Response(status_code=200 if app.state.ready else 503)

Set Kubernetes readinessProbe to hit /healthz/ready. Pod doesn't get traffic until warm.

Cold start budget

For a typical FastAPI service: target < 500 ms from docker run to "first request served." Hitting that:

  • ~0 ms image pull if the image is already on the node; otherwise seconds, which is why image size matters for scale-out.
  • ~100 ms container setup.
  • ~200 ms Python interpreter + imports (with precompiled .pyc).
  • ~50 ms app startup (DB connection, warm-up).

Above 500 ms, you're losing autoscaling responsiveness. Above 2 seconds, you're losing user experience during scale events.


15. ASGI/WSGI Server Choice: uvicorn, gunicorn, granian, hypercorn

For HTTP/web services in containers, the server choice affects memory, CPU, and concurrency model.

Server Sync/Async Multi-process Use case
gunicorn WSGI (sync) Yes (--workers) Django, Flask, classic WSGI apps
uvicorn ASGI (async) Single process (use --workers carefully) FastAPI, Starlette, async Django
gunicorn + uvicorn_worker.UvicornWorker ASGI (async) Yes (gunicorn manages, uvicorn handles) FastAPI with several processes per pod
granian ASGI/WSGI/RSGI (Rust-backed) Yes Performance-critical Python web apps
hypercorn ASGI Yes HTTP/2, HTTP/3 needs
daphne ASGI Yes Django Channels (WebSockets)

The right default for FastAPI on Kubernetes: one uvicorn (or granian) process per pod, scaled by replicas (see below). Use gunicorn with uvicorn workers when you need several processes per pod: --preload memory sharing, or isolation between workers. The worker class moved out of uvicorn: uvicorn.workers is deprecated since uvicorn 0.30, so install the uvicorn-worker package and use uvicorn_worker.UvicornWorker.

ENTRYPOINT ["gunicorn", \
    "-k", "uvicorn_worker.UvicornWorker", \
    "--workers", "4", \
    "--worker-tmp-dir", "/dev/shm", \
    "--bind", "0.0.0.0:8000", \
    "--access-logfile", "-", \
    "--preload", \
    "app.main:app"]

Notes:

  • --worker-tmp-dir /dev/shm: gunicorn writes worker heartbeat files. By default /tmp, which may be on the container's overlay disk; /dev/shm (tmpfs) avoids heartbeat stalls when the disk is slow. With readOnlyRootFilesystem: true, /tmp isn't writable unless you mount an emptyDir there, and gunicorn won't start without this flag or that mount.
  • --workers 4: usually 2 * CPU + 1, where CPU means the pod's limit, never os.cpu_count() (§21). In containers, cap based on memory and request profile. Async workers can handle hundreds of concurrent requests each, so don't over-provision workers.
  • --preload: master imports app, forks workers. COW memory savings (see §13).
  • --access-logfile -: log to stdout.

For pure performance, granian is faster than gunicorn+uvicorn:

ENTRYPOINT ["granian", "--interface", "asgi", "--workers", "4", "--host", "0.0.0.0", "--port", "8000", "app.main:app"]

Granian is Rust-based, handles HTTP at the C/Rust layer, dispatches into Python only for the application code. ~20–40% lower CPU per request on benchmarks.

Single uvicorn vs gunicorn+uvicorn

In Kubernetes, a common pattern is one uvicorn per pod, scale by replica count:

uvicorn app.main:app --host 0.0.0.0 --port 8000
  • Simpler than gunicorn+uvicorn.
  • Each pod is one process; no preload, no master.
  • Scale concurrency horizontally (more pods, fewer workers per pod).
  • Memory is predictable per pod.
  • Plays well with Kubernetes autoscaling.

This is increasingly the default for async services. Use gunicorn+uvicorn when you want process-level isolation within a pod (one bad worker crashes, others survive).

Sync Django

Django (even with async views in 4+) is mostly sync. Use gunicorn with sync workers:

gunicorn --workers 4 --threads 2 --worker-class gthread --preload app.wsgi

gthread (gunicorn's threaded sync worker) handles I/O-bound DB queries with threads. Pure sync workers block on the DB.


16. __pycache__ Strategies and Read-Only Filesystems

Two consistent strategies for __pycache__:

Strategy A: Precompile at build, ban writes at runtime.

ENV UV_COMPILE_BYTECODE=1 \
    PYTHONDONTWRITEBYTECODE=1
securityContext:
  readOnlyRootFilesystem: true

This is the recommended pattern. Bytecode is precompiled, sits in the image, runtime never writes. Read-only FS prevents an attacker from writing a webshell. Cold start is fast.

Strategy B: Allow writes, write at first import.

# Don't set PYTHONDONTWRITEBYTECODE at all: any non-empty value, even "0", disables writes.
ENV PYTHONPYCACHEPREFIX=/var/cache/pycache
volumeMounts:
  - name: pycache
    mountPath: /var/cache/pycache
volumes:
  - name: pycache
    emptyDir: {}

PYTHONPYCACHEPREFIX (3.8+) redirects every .pyc write to a separate tree, so the root filesystem can stay read-only. Don't mount the emptyDir over site-packages: an empty volume there hides every installed package.

Not recommended in production. The first request pays the compilation cost. Subsequent requests (same pod) are fast. New pods start cold.

Strategy A wins. The "compile at runtime" strategy was a workaround for build pipelines that couldn't precompile; with uv, you can.

If you must use Strategy B (e.g., dynamic plugins loaded at runtime), at least don't ship a read-only filesystem.


17. Reproducibility with uv lock

uv.lock is hashed and deterministic. Two builds from the same lockfile produce identical site-packages directories.

For bit-identical image digests (advanced, see ch 39 §13):

ENV UV_COMPILE_BYTECODE=0 \
    UV_LINK_MODE=copy

RUN --mount=type=cache,target=/root/.cache/uv \
    uv sync --locked --no-dev --no-install-project
# Hash-based .pyc: content depends only on the source, not on install-time mtimes.
RUN .venv/bin/python -m compileall -q -j 0 --invalidation-mode unchecked-hash .venv/lib
SOURCE_DATE_EPOCH=$(git log -1 --format=%ct) \
  docker buildx build --output type=image,name=app,rewrite-timestamp=true .

Two separate problems. Inside a timestamp-mode .pyc, the header embeds the source file's mtime, which differs per install; hash-based .pycs embed a hash of the source instead, so they're identical across builds, and unchecked-hash also skips the freshness check at import (§25). Around the files, every layer tarball records file mtimes; BuildKit's rewrite-timestamp=true clamps them to SOURCE_DATE_EPOCH. Don't put SOURCE_DATE_EPOCH in an ENV line: it flips py_compile's default to checked-hash, which hashes every source file on every import (§25), and touching the .pyc files afterwards changes neither problem.

Most teams don't need bit-identical images. They need dependency-identical images, which uv.lock + --locked gives you for free.

Lockfile hygiene

  • Commit uv.lock to git. Always.
  • Make sure .dockerignore does not exclude uv.lock (the build needs it). It should exclude .venv/, __pycache__/, .git/, .env, and test caches; see §27.
  • Run uv lock --upgrade periodically (Renovate and Dependabot both understand uv.lock) to absorb security fixes.
  • In CI, fail fast if uv.lock is stale:
- name: Verify lockfile
  run: uv lock --check        # fails if pyproject.toml needs uv.lock to change

This catches the bug where a contributor added a dependency to pyproject.toml but forgot to uv lock.


18. CI/CD Patterns for uv Builds

GitHub Actions

name: Build
on: [push]
jobs:
  build:
    runs-on: ubuntu-latest
    permissions: { contents: read, packages: write }
    steps:
      - uses: actions/checkout@v4

      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v3

      - name: Log in to GHCR
        uses: docker/login-action@v3
        with:
          registry: ghcr.io
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}

      - name: Build and push
        uses: docker/build-push-action@v6
        with:
          context: .
          push: true
          tags: |
            ghcr.io/${{ github.repository }}:${{ github.sha }}
            ghcr.io/${{ github.repository }}:latest
          cache-from: type=gha
          cache-to: type=gha,mode=max
          provenance: true
          sbom: true
          platforms: linux/amd64,linux/arm64

cache-to type=gha,mode=max exports all stages' layers; cache-from type=gha restores them. On hosted runners that layer cache is what makes rebuilds fast: the uv cache mount starts empty on every fresh runner and is not exported (§27). With an unchanged uv.lock, the dependency layer is a cache hit and the build takes ~30 seconds; after a lockfile change, it re-downloads everything. Note also that linux/arm64 on an amd64 runner builds under QEMU emulation, which is 5–20× slower; use a native arm64 runner per platform when builds get long.

Testing in the same Dockerfile

FROM ... AS test
RUN --mount=type=cache,target=/root/.cache/uv \
    uv sync --locked
COPY tests ./tests
RUN uv run pytest -q --tb=short

FROM ... AS build
# ... production stages
docker buildx build --target test .          # CI: tests run
docker buildx build --target build .         # Deploy: tests are skipped (separate target)

--target test builds up to the test stage and stops. --target build skips the test stage entirely. CI runs both: tests first (gate), then build (the artifact).

A more concise variant — tests inside the build stage:

FROM ... AS builder
COPY pyproject.toml uv.lock ./
RUN --mount=type=cache,target=/root/.cache/uv \
    uv sync --locked

COPY src ./src
COPY tests ./tests
RUN --mount=type=cache,target=/root/.cache/uv \
    uv run pytest -q

RUN --mount=type=cache,target=/root/.cache/uv \
    uv sync --locked --no-dev    # remove dev deps for the final venv

Tests run as part of the build; failures abort the build. uv sync is exact by default, so the last step removes the dev dependencies. Don't then copy the whole /app to the runtime stage: tests/ and pytest's caches are still there. Copy only .venv and src.


19. The Gold-Standard Dockerfile, Fully Annotated

Putting everything together for a typical FastAPI service:

# syntax=docker/dockerfile:1.7
# ============================================================================
# Stage 1: Builder
#   - Astral's uv image (Debian slim + uv); pin it by digest in real use.
#   - Installs a uv-managed CPython into /python, so builder and runtime run
#     the exact same interpreter binary (no distroless 3.11-vs-3.12 mismatch).
#   - Installs deps with cache mount, compiles bytecode at install time.
#   - Strips tests, docs, locales from site-packages.
# ============================================================================
FROM ghcr.io/astral-sh/uv:bookworm-slim AS builder

ENV UV_COMPILE_BYTECODE=1 \
    UV_LINK_MODE=copy \
    UV_PYTHON_INSTALL_DIR=/python \
    UV_PYTHON_PREFERENCE=only-managed \
    UV_PYTHON=3.12 \
    PYTHONDONTWRITEBYTECODE=1 \
    PYTHONUNBUFFERED=1

# binutils only for `strip` below; the builder stage never ships.
RUN apt-get update && apt-get install -y --no-install-recommends binutils \
 && rm -rf /var/lib/apt/lists/*
RUN uv python install 3.12

WORKDIR /app

# Dependency layer
COPY pyproject.toml uv.lock ./
RUN --mount=type=cache,target=/root/.cache/uv \
    uv sync --locked --no-dev --no-install-project

# Project layer: non-editable, so the app is installed (and compiled) into the venv
COPY src ./src
COPY README.md ./
RUN --mount=type=cache,target=/root/.cache/uv \
    uv sync --locked --no-dev --no-editable

# Slim the venv
RUN find /app/.venv -type d -name tests -prune -exec rm -rf {} + \
 && find /app/.venv -type d -name "*.dist-info" -exec sh -c \
        'for d; do rm -f "$d/RECORD" "$d/INSTALLER" "$d/REQUESTED"; done' _ {} \; \
 && find /app/.venv -name "*.so" -exec strip --strip-unneeded {} + 2>/dev/null || true \
 && find /app/.venv -path "*/locale/*" \( -name "*.mo" -o -name "*.po" \) -delete

# Slim the interpreter: no pip, IDLE, or Tk at runtime (~20 MB).
RUN find /python \( -name ensurepip -o -name idlelib -o -name tkinter -o -name turtledemo \
        -o -name pip -o -name 'pip-*.dist-info' -o -name 'tcl9*' -o -name 'tk9*' \
        -o -name 'libtcl*' -o -name '_tkinter*.so' \) -prune -exec rm -rf {} +

# ============================================================================
# Stage 2: Test (optional, run via --target test)
# ============================================================================
FROM builder AS test

RUN --mount=type=cache,target=/root/.cache/uv \
    uv sync --locked --no-editable   # bring in dev deps; tests import the installed `app`
COPY tests ./tests
RUN uv run --no-sync pytest -q tests/

# ============================================================================
# Stage 3: Runtime
#   - Distroless cc (glibc + libstdc++, no Python), nonroot.
#   - Only the interpreter and the built venv (which contains the app).
#   - No shell, no package manager.
# ============================================================================
FROM gcr.io/distroless/cc-debian12:nonroot AS runtime

# No PYTHONOPTIMIZE: the .pyc files above are level 0, and -O would ignore them (§25).
ENV PYTHONDONTWRITEBYTECODE=1 \
    PYTHONUNBUFFERED=1 \
    PYTHONHASHSEED=random \
    MALLOC_ARENA_MAX=2 \
    PATH="/app/.venv/bin:${PATH}"

WORKDIR /app

COPY --from=builder /python /python
COPY --from=builder --chown=nonroot:nonroot /app/.venv /app/.venv

USER nonroot
EXPOSE 8000

# Bind to 0.0.0.0 inside the pod; service maps the port.
ENTRYPOINT ["python", "-m", "uvicorn", "app.main:app", \
            "--host", "0.0.0.0", "--port", "8000", \
            "--no-access-log"]

Annotations:

  • Cache mounts on the global uv cache mean repeated builds reuse downloaded wheels.
  • --no-install-project in the first sync gives a clean dependency layer; project install in the second sync is fast and re-runs only when source changes.
  • uv run --no-sync in the test stage runs pytest against the environment exactly as synced, instead of letting uv run re-sync (and switch the project back to editable).
  • --locked ensures uv.lock is the source of truth (build fails if it's stale).
  • Strip tests/dist-info/locales trims ~30–80 MB.
  • strip --strip-unneeded on .so files removes debug info (needs binutils, installed in the builder only).
  • Slim the interpreter: the managed CPython is ~120 MB uncompressed (the python3.12 binary is statically linked, ~30 MB, and ships next to a ~30 MB libpython), about twice Debian's. Deleting pip, IDLE, and Tk recovers ~20 MB. That size is the price of choosing your Python version; don't strip the interpreter itself (it's already stripped; ~3 MB gain).
  • UV_PYTHON=3.12 pins the interpreter for both uv python install and uv sync, so a requires-python = ">=3.12" can't make uv download a newer version behind your back.
  • uv-managed interpreter on distroless/cc: the venv's python symlink resolves to /python/... in both stages, so the .pyc version tag and the binary always match. python3-debian12 would give you Debian's 3.11 instead (§25).
  • --no-editable installs the project into site-packages like a dependency: UV_COMPILE_BYTECODE compiles it, the runtime copies one directory, and import app never depends on the working directory (§25). Project layout is shown in §5.
  • Distroless nonroot runtime; no shell, no pip, no curl. Smallest reasonable surface.
  • MALLOC_ARENA_MAX=2 for memory hygiene.
  • No PYTHONOPTIMIZE: it would make the interpreter ignore every level-0 .pyc we just compiled (§25).
  • PYTHONHASHSEED=random explicit (defensive, also the default).
  • Single uvicorn process per pod, scale via replicas.
  • --no-access-log for low CPU overhead; ship access logs from a sidecar if needed.

Expected result for a FastAPI service with ~30 deps:

  • Image size: 70–90 MB compressed. (Measured for this exact Dockerfile with only fastapi + uvicorn: 45 MB compressed, 192 MB uncompressed; import app.main in ~0.4 s from a read-only root, vs ~1.2 s with -O, which ignores the level-0 .pycs.)
  • Cold start (from docker run to first response): ~300–500 ms.
  • RSS under idle: ~80 MB.
  • RSS under load: ~150–200 MB.

Compare to the "naive" version (python:3.12 + pip install):

  • Image size: ~450–550 MB compressed (over 1 GB uncompressed).
  • Cold start: ~1.5–2.5 s.
  • RSS under idle: ~120 MB.
  • RSS under load: ~200–300 MB.

4–10× smaller image, 3–5× faster cold start, 20–30% lower RSS. Same code.

The matching pod spec

The image assumes a pod that is read-only, non-root, and tells Python how much CPU and memory it really has (§21, §22):

apiVersion: apps/v1
kind: Deployment
metadata: { name: api }
spec:
  replicas: 3
  selector: { matchLabels: { app: api } }
  template:
    metadata: { labels: { app: api } }
    spec:
      automountServiceAccountToken: false
      securityContext:
        runAsNonRoot: true
        runAsUser: 65532
        seccompProfile: { type: RuntimeDefault }
      terminationGracePeriodSeconds: 30
      containers:
        - name: api
          image: ghcr.io/acme/api@sha256:...      # pin by digest
          ports: [{ containerPort: 8000 }]
          env:
            - name: PYTHON_CPU_COUNT
              valueFrom: { resourceFieldRef: { resource: limits.cpu, divisor: "1" } }
            - name: OMP_NUM_THREADS
              valueFrom: { resourceFieldRef: { resource: limits.cpu, divisor: "1" } }
            - name: MEMORY_LIMIT_BYTES
              valueFrom: { resourceFieldRef: { resource: limits.memory } }
          resources:
            requests: { cpu: 500m, memory: 192Mi }
            limits:   { cpu: "1", memory: 384Mi }
          securityContext:
            readOnlyRootFilesystem: true
            allowPrivilegeEscalation: false
            capabilities: { drop: [ALL] }
          startupProbe:
            httpGet: { path: /healthz, port: 8000 }
            periodSeconds: 1
            failureThreshold: 30                  # measured under the CPU limit, not on a laptop
          readinessProbe:
            httpGet: { path: /healthz/ready, port: 8000 }
            periodSeconds: 5
          livenessProbe:
            httpGet: { path: /healthz, port: 8000 }
            periodSeconds: 10
          lifecycle:
            preStop: { sleep: { seconds: 5 } }   # let endpoints drain before SIGTERM (§24)
          volumeMounts:
            - { name: tmp, mountPath: /tmp }
      volumes:
        - name: tmp
          emptyDir: { sizeLimit: 64Mi }

20. Measuring: How to Tell If You Actually Improved Anything

Don't optimize without measurement. The relevant numbers:

Image size

docker images myapp:latest --format "{{.Size}}"
docker inspect myapp:latest | jq '.[0].Size'      # uncompressed
docker image history myapp:latest
dive myapp:latest                                   # layer-by-layer with score

For a quick check that you're not adding bloat: dive shows "wasted space" — files added and later removed, which still occupy bytes.

Cold start

time docker run --rm --read-only --entrypoint python myapp:latest -c "from app.main import app"

--entrypoint is required because the image's ENTRYPOINT would otherwise receive python -c ... as arguments; --read-only reproduces the production filesystem, so missing .pycs show up as slow imports. This roughly captures import time. For real cold start, time from pod creation to Ready, with a readiness probe that hits a real endpoint (without one, the pod is Ready the moment the process starts, and you measure nothing):

# In a test cluster
# pod.yaml: the image's own ENTRYPOINT, plus an httpGet readinessProbe on /healthz/ready
START=$(date +%s.%N)
kubectl apply -f pod.yaml
kubectl wait --for=condition=Ready pod/coldstart --timeout=120s
END=$(date +%s.%N)
echo "Cold start: $(echo "$END - $START" | bc) seconds"
kubectl get events --field-selector involvedObject.name=coldstart   # Pulling -> Pulled = pull time

Memory

# Inside the pod
cat /sys/fs/cgroup/memory.current      # cgroup v2
cat /sys/fs/cgroup/memory/memory.usage_in_bytes   # cgroup v1

# From Python
import resource; resource.getrusage(resource.RUSAGE_SELF).ru_maxrss   # in KB

Memory profiling tools: memray (the modern choice: tracks native allocations too, low overhead, good flame graphs) and tracemalloc (stdlib, Python allocations only). py-spy is a CPU sampling profiler, not a memory one, but it answers the companion question: where startup time goes (py-spy record -- python -m yourapp). ru_maxrss is the peak RSS (KiB on Linux); for the number the OOM killer uses, read memory.current and memory.peak (cgroup v2).

Import time

python -X importtime -c "import app.main" 2> imports.log
# Slowest 20 by cumulative time (columns: self | cumulative | package):
sort -t'|' -k2 -n imports.log | tail -20
# Or visualize the tree:
uvx tuna imports.log

Anything above 50 ms per import deserves investigation.

Continuous tracking

In production:

  • Image size in CI: fail the build if size grows >20% week-over-week.
  • Cold start in observability: tag pod start events with a histogram bucket, alert on regressions.
  • RSS as a standard cluster metric: alert when p99 approaches memory limit.

Don't trust "I made it smaller" without numbers. Don't trust "it's faster now" without a probe time histogram.


21. CPU: What Python Thinks It Has vs What the cgroup Gives It

The most expensive surprise in containerized Python is a number: os.cpu_count(). On a 64-core node, inside a pod with limits.cpu: 2, it returns 64. A CPU limit is a CFS bandwidth quota (cpu.max), not a set of CPUs; every core stays visible, you just get throttled once the cgroup has burned 2 CPU-seconds per second.

Three different questions, three different answers:

API What it reads Respects cpuset / affinity Respects CFS quota (limits.cpu)
os.cpu_count() CPUs online on the host No No
len(os.sched_getaffinity(0)) CPUs this process may be scheduled on Yes No
os.process_cpu_count() (3.13+) Same as affinity Yes No
PYTHON_CPU_COUNT=N / -X cpu_count=N (3.13+) Whatever you set Overrides all three above Overrides all three above

Affinity only shrinks when the kubelet's static CPU manager policy pins a Guaranteed-QoS pod with integer CPUs to dedicated cores. In every other pod, affinity is the whole node too. CPython deliberately does not parse cgroups (gh-109595): the spec is complex and the answer can change at runtime. So the only reliable answer is the one you tell it.

Who calls it behind your back

Consumer Default sizing Knob
ThreadPoolExecutor (incl. asyncio's default executor, so every getaddrinfo and to_thread) min(32, cpu + 4) max_workers=, or PYTHON_CPU_COUNT
ProcessPoolExecutor, multiprocessing.Pool cpu processes max_workers= / processes=
gunicorn / uvicorn workers 1 unless set; the classic formula from gunicorn's docs, 2 × cpu + 1, is usually implemented as multiprocessing.cpu_count() * 2 + 1 WEB_CONCURRENCY (both honor it), --workers
NumPy/SciPy via OpenBLAS, MKL; scikit-learn one thread per visible core OPENBLAS_NUM_THREADS, MKL_NUM_THREADS, OMP_NUM_THREADS
PyTorch intra-op threads physical cores torch.set_num_threads(), OMP_NUM_THREADS
Polars, PyArrow, DuckDB visible cores POLARS_MAX_THREADS, pyarrow.set_cpu_count(), SET threads
glibc malloc arenas 8 × cores MALLOC_ARENA_MAX (§13)

The failure mode is not a crash. It is CFS throttling: 64 OpenBLAS threads on a 2-CPU quota burn the whole 100 ms period's budget in ~3 ms, then the cgroup is frozen for the remaining ~97 ms. p50 looks fine, p99 has 100 ms steps in it, and container_cpu_cfs_throttled_periods_total climbs. A ProcessPoolExecutor() with no arguments on that same node forks 64 workers, each with its own copy of your imports, and the pod OOMs instead.

Measured: the quota is invisible, the throttling is not

Python 3.13.16, process in a cgroup with cpu.cfs_quota_us = 50000 (cgroup v1; cpu.max = 50000 100000 on v2), i.e. 0.5 CPU, on a 2-core host:

  • os.cpu_count() = 2, os.process_cpu_count() = 2. The quota is invisible to both.
  • The quota is real: a CPU-bound loop took 5.6 s wall for 2.8 s of CPU, with nr_throttled = 56 (one throttled 100 ms period for almost every period it ran).
  • gunicorn with workers = multiprocessing.cpu_count() * 2 + 1 started 5 Uvicorn workers. Each FastAPI worker: ~68 MB RSS / ~49 MB PSS. Total PSS with master: 264 MB.
  • Extrapolated to a 64-core node: 129 workers × ~49 MB ≈ 6.3 GB, for a pod that was given half a CPU.

Reproduced in Docker (--cpus=0.5, Python 3.13.14, 4-core host, a minimal FastAPI app): cpu_count() = process_cpu_count() = 4; the same loop took 5.6 s wall for 2.8 s CPU with nr_throttled = 57; gunicorn started 9 workers at ~48 MB RSS / ~31 MB PSS each, 300 MB PSS total. Adding --cpuset-cpus=0,1 changed process_cpu_count() to 2 while cpu_count() stayed 4: affinity is visible, quota never is.

Read your own pod's numbers with cat /sys/fs/cgroup/cpu.stat (v2: nr_throttled, throttled_usec) and PSS per worker from /proc/<pid>/smaps_rollup.

Common objections

  • "gunicorn defaults to 1 worker." True. The problem is the formula from its docs, which many configs implement with multiprocessing.cpu_count(); that's the code path measured above.
  • "Use the static CPU manager." It does shrink affinity, so process_cpu_count() becomes correct, but only for Guaranteed pods with whole-CPU requests on nodes configured for it. Every Burstable pod, and every fractional limit, still sees the whole node.
  • "500m rounds up to 1." Yes: the Downward API with divisor: "1" rounds up, so a 500m limit gives PYTHON_CPU_COUNT=1. That's the safe direction: one worker on half a CPU, not zero, not 129.
  • "We're on Python < 3.13." PYTHON_CPU_COUNT doesn't exist there, and is silently ignored. Set WEB_CONCURRENCY and the thread-count variables (OMP_NUM_THREADS, OPENBLAS_NUM_THREADS, MKL_NUM_THREADS, POLARS_MAX_THREADS) explicitly; they work on every Python version. For pools you create in code, size them with effective_cpus() below.

The fix: tell Python the truth via the Downward API

env:
  - name: PYTHON_CPU_COUNT          # 3.13+: fixes os.cpu_count() and every default derived from it
    valueFrom:
      resourceFieldRef:
        resource: limits.cpu
        divisor: "1"                # rounds UP: 500m -> 1, 1500m -> 2
  - name: OMP_NUM_THREADS
    valueFrom:
      resourceFieldRef: { resource: limits.cpu, divisor: "1" }
  - name: OPENBLAS_NUM_THREADS
    valueFrom:
      resourceFieldRef: { resource: limits.cpu, divisor: "1" }
  - name: WEB_CONCURRENCY
    value: "1"                      # async service: one process per pod, scale with replicas

Two caveats: if the pod has no CPU limit, limits.cpu resolves to the node's allocatable CPU, so you are back to 64; teams that drop CPU limits (a defensible choice, to avoid throttling) must set these to a fixed number instead. And on Python ≤ 3.12 PYTHON_CPU_COUNT is ignored, so the explicit variables above carry the whole fix; for pools sized in code, read the quota yourself:

import math
import os


def effective_cpus() -> int:
    """CPUs this process can actually burn: min(affinity, cgroup quota)."""
    n = len(os.sched_getaffinity(0))
    try:
        with open("/sys/fs/cgroup/cpu.max") as f:          # cgroup v2
            quota, period = f.read().split()
    except OSError:
        try:                                               # cgroup v1
            with open("/sys/fs/cgroup/cpu/cpu.cfs_quota_us") as f:
                quota = f.read().strip()
            with open("/sys/fs/cgroup/cpu/cpu.cfs_period_us") as f:
                period = f.read().strip()
        except OSError:
            return n
    if quota not in ("max", "-1"):
        n = min(n, max(1, math.ceil(int(quota) / int(period))))
    return n

Cold start is CPU-bound, and the limit applies to it

Imports, .pyc unmarshalling, and warm-up are pure CPU. A service that starts in 400 ms with a full core starts in ~1.6 s at limits.cpu: 250m, and the startupProbe you tuned on a laptop fails. Either give requests/limits headroom for startup, set the startupProbe from a measurement taken under the limit, or (Kubernetes 1.33+, in-place pod resize) start with more CPU and resize down after ready.

Deeper: how the scheduler sees threads and processes, ../python-mastery/06-processes-threads-scheduling.md; why the GIL makes extra threads useless for CPU-bound Python but not for BLAS, ../python-mastery/24-the-gil.md.


22. Memory: Limits Python Can't See, OOMKills It Can't Catch

Python sees the host's RAM

os.sysconf("SC_PHYS_PAGES") * os.sysconf("SC_PAGE_SIZE") and psutil.virtual_memory().total report the node's memory, not the cgroup's. Anything that sizes a cache "as 25% of RAM" (some ML loaders, in-process LRU caches, a homegrown batcher) sizes it against 256 GB inside a 1 GB pod. Read the limit, or inject it:

def memory_limit_bytes() -> int | None:
    try:
        with open("/sys/fs/cgroup/memory.max") as f:
            v = f.read().strip()
        return None if v == "max" else int(v)
    except OSError:
        return None
env:
  - name: MEMORY_LIMIT_BYTES
    valueFrom:
      resourceFieldRef: { resource: limits.memory }

MemoryError will not save you

When the cgroup hits memory.max, the kernel OOM killer sends SIGKILL. No exception, no traceback, no atexit, no flushed logs; the container exits 137 and kubectl describe says OOMKilled. MemoryError only happens when a single malloc fails, which in a cgroup almost never happens first. If you want a graceful failure, it has to be a self-imposed limit below the cgroup's: reject work when memory.current crosses ~85% of memory.max, or recycle the worker (below).

What the kubelet actually counts

  • memory.current includes page cache your process caused (files read, logs written to a file in the container, tmpfs).
  • The kubelet evicts and the dashboards alert on working set = memory.current − inactive_file. That is the number to compare against the limit, not ps RSS.
  • RSS double-counts shared pages across processes. With 4 gunicorn workers sharing a preloaded app, summing RSS over-reports. Use PSS (/proc/<pid>/smaps_rollup) per process and memory.current for the pod.
  • emptyDir: { medium: Memory } and /dev/shm writes are RAM charged to the pod.

Fragmentation: RSS that never comes back

Python almost never returns memory to the OS, for two independent reasons:

  1. pymalloc (objects ≤ 512 bytes) carves 1 MiB arenas (3.10+) into 16 KiB pools. An arena is released only when every object in it is freed. One long-lived dict entry pins the megabyte.
  2. glibc malloc (everything bigger, plus every C extension) only trims the top of the heap; holes in the middle stay resident. Per-thread arenas multiply this (§13).

The levers, cheapest first:

  • MALLOC_ARENA_MAX=2 (already in §13).
  • Recycle workers: gunicorn --max-requests 2000 --max-requests-jitter 200. Crude, bounded, works for any leak or fragmentation source. The jitter stops all workers restarting at once.
  • Swap the allocator for C-extension-heavy apps: LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libjemalloc.so.2 (copy the .so into a distroless image from the builder). jemalloc's decay returns memory on its own; typical RSS reduction is 10–30% on pandas/numpy-heavy services. Measure; it occasionally costs CPU. CPython's free-threaded build already uses bundled mimalloc.
  • ctypes.CDLL("libc.so.6").malloc_trim(0) after a known large, transient allocation (a batch job, a big report). A targeted hammer, not a default.

Diagnosis ladder and allocator internals: ../python-mastery/08-allocators.md (§5 on arenas, §6 on why memory doesn't come back, §12 on installing jemalloc/mimalloc); RSS vs PSS vs working set: ../python-mastery/07-virtual-memory.md; cutting RSS you can explain: ../python-mastery/35-memory-optimization.md.

Transparent huge pages

If the node runs THP in always mode, sparse heaps get backed by 2 MiB pages and RSS jumps for no code change. Most distros default to madvise, which is fine. If you see unexplained RSS growth after a node image upgrade, check /sys/kernel/mm/transparent_hugepage/enabled before blaming the app.


23. Fork, Copy-on-Write, and the Garbage Collector

§13 promised that gunicorn --preload shares memory across workers via copy-on-write. In CPython that promise leaks, because reading an object writes to it: every reference taken increments ob_refcnt in the object header, dirtying its page, and the kernel copies the page into the child. A full GC pass is worse: it walks every tracked object and writes GC bookkeeping into each one. A preloaded Django app that shares 80% of its pages right after fork can be down to 20% shared an hour later.

What helps:

  • Python 3.12+ immortal objects (PEP 683): None, True, small ints, interned strings, and many static objects no longer have their refcount written. Free improvement from upgrading.
  • gc.freeze() (3.7+) moves every object currently tracked into a permanent generation the collector never scans, so GC in the children doesn't dirty the parent's pages. The documented recipe is disable early in the parent, freeze right before fork, re-enable in the child:
# gunicorn.conf.py
import gc

preload_app = True
max_requests = 2000
max_requests_jitter = 200

gc.disable()                      # master: no collections creating holes while importing


def pre_fork(server, worker):     # runs in the master, immediately before each fork
    gc.freeze()


def post_fork(server, worker):    # runs in each worker
    gc.enable()

Measure with the sum of Shared_* in /proc/<worker>/smaps_rollup, before and after an hour of traffic. Instagram, who upstreamed gc.freeze(), reported a measurable capacity gain per host from it.

The multiprocessing start method changed under you

  • Python 3.12 emits a DeprecationWarning when you fork() a process that has threads, because the child inherits locks held by threads that no longer exist (the classic logging/malloc deadlock).
  • Python 3.14 changed the default start method on Linux from fork to forkserver. Code that relied on children inheriting a preloaded model, an open DB pool, or module globals now pays a fresh import per worker, and anything passed to a worker must be picklable. If you depend on fork semantics, say so: multiprocessing.get_context("fork"), and accept the threads caveat.

The fork mechanics, the deadlock reproduced, and the 3.14 change: ../python-mastery/10-signals-fork-exec.md (§8–§10). Why refcounting writes on read: ../python-mastery/15-refcounting-and-ownership.md. The collector walked line by line: ../python-mastery/22-garbage-collection.md.


24. PID 1, Signals, and Graceful Shutdown

Whatever ENTRYPOINT starts is PID 1 in the container's PID namespace, and the kernel treats PID 1 specially: a signal whose disposition is the default is dropped, even when sent from outside (only SIGKILL/SIGSTOP are forced). CPython installs a handler for SIGINT but not for SIGTERM. So:

Entrypoint What kubectl delete pod does
python worker.py, no SIGTERM handler SIGTERM ignored → nothing happens for terminationGracePeriodSeconds (30 s) → SIGKILL. Every rollout takes 30 s per pod and in-flight work is cut.
CMD python app.py (shell form) /bin/sh -c is PID 1 and doesn't forward signals; same 30 s, and Python never hears about it. In distroless there is no shell, so the container doesn't even start.
uvicorn / gunicorn / granian (exec form) They install SIGTERM handlers: stop accepting, drain, exit. Correct.
Anything that spawns subprocesses PID 1 must also reap zombies. Python only reaps children it wait()s on.

Rules:

  1. Always exec form (ENTRYPOINT ["python", "-m", ...]).
  2. For plain scripts and job runners, handle SIGTERM explicitly:
import signal
import sys

def _terminate(signum, frame):
    raise SystemExit(0)           # unwinds: finally blocks, context managers, atexit run

signal.signal(signal.SIGTERM, _terminate)
  1. If the process spawns children it doesn't reap, run under an init: docker run --init, shareProcessNamespace, or copy tini into the image and ENTRYPOINT ["/tini", "--", "python", ...].
  2. Draining is not just the signal. Endpoint removal propagates to kube-proxy/ingress after SIGTERM is sent, so a pod that exits instantly still drops requests. Add lifecycle.preStop: { sleep: { seconds: 5 } } (Kubernetes 1.30+ native sleep; no shell needed) and keep the server's graceful timeout below terminationGracePeriodSeconds.

How CPython defers handlers to the eval loop (and why a long C call delays your SIGTERM): ../python-mastery/10-signals-fork-exec.md (§3, §4, §13 on reaping, §15 on graceful shutdown assembled).


25. Bytecode Deep Cuts: Optimization Levels, Invalidation Modes, Version Tags

§8 says "flip UV_COMPILE_BYTECODE=1 and you're done." Three ways it silently isn't:

Surprise 1: PYTHONOPTIMIZE looks for different files

.pyc files are tagged by optimization level:

__pycache__/models.cpython-312.pyc          # level 0 (default)
__pycache__/models.cpython-312.opt-1.pyc    # -O  / PYTHONOPTIMIZE=1
__pycache__/models.cpython-312.opt-2.pyc    # -OO / PYTHONOPTIMIZE=2

uv --compile-bytecode (like pip) compiles at level 0 only. Run with PYTHONOPTIMIZE=1 and the interpreter looks for .opt-1.pyc, doesn't find it, and compiles from source. With PYTHONDONTWRITEBYTECODE=1 and a read-only root filesystem it can't cache the result, so every start pays the full compile: you've shipped the .pycs and switched them off. Pick one:

  • Run at level 0 (what the gold-standard Dockerfile in §19 now does). The startup win from -O is small; asserts are cheap.
  • Or compile everything at the level you run: python -m compileall -q -j 0 -o 1 /app/.venv/lib (with a non-editable install the app is in there too; repeat -o for several levels).

Check it in the built image: python -X importtime -c 'import yourapp' 2>&1 | tail -1 should be identical with and without --read-only.

Surprise 2: your own code isn't compiled

uv sync installs your project editable by default: site-packages holds a .pth pointing at /app/src, and --compile-bytecode compiles site-packages, not /app/src. The dependencies start fast; your app's own modules compile on every start. Use uv sync --no-editable (the project becomes a normal installed package, compiled with the rest, and only .venv needs copying to the runtime stage; this is what §5 and §19 do). If you must keep an editable install, compileall the source tree yourself:

Surprise 3: invalidation modes, and SOURCE_DATE_EPOCH

How a .pyc proves it is fresh (PEP 552):

Mode Check on every import Cost Use
timestamp (default) stat() the .py, compare mtime + size one syscall per module dev
checked-hash read and hash the whole .py full source read + SipHash per module rarely
unchecked-hash nothing zero immutable images

Two traps. Setting SOURCE_DATE_EPOCH (common in reproducible-build pipelines) silently flips py_compile's default to checked-hash, so startup gets slower the moment you make builds reproducible. And anything that rewrites mtimes after compilation (BuildKit's rewrite-timestamp=true, some image-squashing tools, COPY from a stage that touched files) makes timestamp .pycs stale, and they're recompiled at runtime. In an image, sources never change after build, so the correct mode is:

RUN python -m compileall -q -j 0 --invalidation-mode unchecked-hash /app/src

Surprise 4: the version tag must match the runtime interpreter

cpython-312 in the filename is the interpreter's magic number. Compile with 3.12 in the builder and run on a 3.11 runtime, and every .pyc is ignored (and the venv's python symlink points at a binary that doesn't exist, so it won't start at all). This is the classic distroless trap: gcr.io/distroless/python3-debian12 ships Debian's Python, which is 3.11, not whatever your builder used. §19 fixes it by shipping a uv-managed interpreter on distroless/cc, so builder and runtime run the same binary by construction.

Smaller dials

  • Frozen stdlib (3.11+): core startup modules are deep-frozen into the binary; that's why bare python -c pass is ~15 ms. Don't disable it (-X frozen_modules=off is for debugging the stdlib).
  • Lazy imports in the language: PEP 810 (lazy import x, accepted for Python 3.15) makes §14's hand-written lazy imports a keyword. Until you're on it, the function-local import and PEP 562 __getattr__ patterns stand.
  • zipapps (shiv, pex): one file, convenient for CLIs, but imports from a zip can't use cached .pyc unless the archive includes them, and C extensions must be extracted at first run. For services, a venv in an image is faster.

What a code object and a .pyc actually contain: ../python-mastery/19-bytecode-and-code-objects.md.


26. Poetry vs uv (vs pip) in Containers

Poetry is the most common thing teams are migrating from. It's a reasonable tool; the question is what it costs in a Dockerfile.

pip + requirements.txt Poetry 2.x uv
Lockfile none native; pip-compile/uv pip compile for a hashed, per-platform file poetry.lock, cross-platform, hashed uv.lock, cross-platform, hashed
Resolve speed (cold, ~100 deps) n/a tens of seconds ~1 s
Install speed serial download+install parallel, Python-speed parallel, Rust-speed, 10–100× pip
Tool footprint in the builder already there Poetry + ~40 deps (~70 MB), must be pinned one static binary, COPY --from=ghcr.io/astral-sh/uv
Manages the Python interpreter no no yes (uv python install)
BuildKit cache-mount pattern --mount=type=cache,target=/root/.cache/pip POETRY_CACHE_DIR + mount /root/.cache/uv + UV_LINK_MODE=copy
Bytecode at install --compile (default on for pip) not by default; compileall after --compile-bytecode
PEP 621 [project] table n/a yes, since 2.0 yes
Export to requirements.txt n/a poetry-plugin-export (unbundled since 2.0) uv export
Plugin ecosystem, publishing workflows n/a mature thinner

If you're staying on Poetry, do it right

FROM python:3.12-slim-bookworm AS builder
ARG POETRY_VERSION=2.1.3
ENV POETRY_NO_INTERACTION=1 \
    POETRY_VIRTUALENVS_IN_PROJECT=1 \
    POETRY_CACHE_DIR=/tmp/poetry-cache
RUN pip install --no-cache-dir "poetry==${POETRY_VERSION}"   # builder only, never in runtime

WORKDIR /app
COPY pyproject.toml poetry.lock ./
RUN --mount=type=cache,target=/tmp/poetry-cache \
    poetry install --only main --no-root --no-directory       # dependency layer
COPY src ./src
RUN poetry install --only main \
 && .venv/bin/python -m compileall -q -j 0 --invalidation-mode unchecked-hash .venv/lib src

The common mistakes: POETRY_VIRTUALENVS_CREATE=false (installs into the system site-packages alongside Poetry's own dependencies, which then ship in the image and can conflict with yours), installing Poetry in the runtime stage, poetry install without --no-root before the source is copied (busts the dependency layer on every code change), and forgetting poetry check --lock in CI so a stale lock reaches the build.

Recommendation and tradeoff

New services: uv. The resolver and installer are an order of magnitude faster, the lockfile is equivalent, and uv manages the interpreter, which removes the builder/runtime version-mismatch class of bug (§25). Existing Poetry repos: the migration is mechanical (uvx migrate-to-uv rewrites pyproject.toml and generates uv.lock); do it when build time or CI minutes hurt. If the team depends on Poetry plugins or its publish flow, keeping Poetry for development and using poetry export → uv pip install --require-hashes in the Dockerfile gets most of the speed without changing developer habits, at the cost of two tools and an export step to keep in sync.


27. Build Cache Deep Cuts

§6 covered the uv cache mount. What bites in real pipelines:

  • Cache mounts don't travel. --mount=type=cache lives on the builder's local disk. --cache-to type=registry / type=gha export layers, never cache mounts. On ephemeral CI runners the mount is empty on every run; the layer cache (lockfile-first ordering, §5) is what actually saves you there. Use --cache-to type=gha,mode=max (or type=registry,mode=max) so the intermediate dependency layer is exported, not just the final stage.
  • Without a mount, the cache ships. RUN uv sync with no cache mount writes hundreds of MB of wheels into /root/.cache/uv in that layer. Either mount it or pass --no-cache. Verify with dive.
  • Concurrent builds on one builder can race on the same cache. --mount=type=cache,target=/root/.cache/uv,sharing=locked, or give each Python version its own id=uv-py312.
  • Non-root builds: if the stage runs as a non-root USER, the mount is owned by root unless you set uid=/gid= on it, and uv falls back to no cache with a warning you won't read.
  • Bind-mount inputs instead of COPYing them when they're only needed for one step; it avoids a layer and keeps the step's cache key exact:
RUN --mount=type=cache,target=/root/.cache/uv \
    --mount=type=bind,source=uv.lock,target=uv.lock \
    --mount=type=bind,source=pyproject.toml,target=pyproject.toml \
    uv sync --locked --no-dev --no-install-project
  • uv cache prune --ci before saving a CI-level cache (e.g. actions/cache on ~/.cache/uv for non-Docker jobs): drops pre-built wheels that are faster to re-download than to restore, keeps the ones built from source.
  • Private package indexes without leaking the token. ARG/ENV values are stored in the image config and docker history. Pass credentials as a BuildKit secret, which exists only for that RUN:
RUN --mount=type=cache,target=/root/.cache/uv \
    --mount=type=secret,id=uv_index_token,env=UV_INDEX_INTERNAL_PASSWORD \
    UV_INDEX_INTERNAL_USERNAME=__token__ \
    uv sync --locked --no-dev --no-install-project
docker buildx build --secret id=uv_index_token,env=INTERNAL_PYPI_TOKEN .

(UV_INDEX_<NAME>_USERNAME / _PASSWORD match an index named internal in [[tool.uv.index]]; the env= form of secret mounts needs Dockerfile syntax 1.10+.) - .dockerignore is part of the cache key. Without it, COPY . . sends .git/, the developer's .venv/ (with host-platform binaries that overwrite the built ones), __pycache__/, and .env secrets into the build, and any change to them busts the layer cache. Minimum: .venv/ **/__pycache__/ .git/ .env .pytest_cache/ .mypy_cache/ .ruff_cache/ dist/ build/. - Base images go stale. Pin bases by digest for reproducibility and let Renovate/Dependabot bump the digest, so Debian and CPython security patches arrive as reviewable PRs instead of never. Scan the result in CI (trivy image, grype); see ch 39 for signing and SBOMs. - Multi-arch: building linux/arm64 under QEMU on an amd64 runner is 5–20× slower and turns any sdist build into minutes. Use native arm64 runners and merge manifests; uv.lock is universal, so the same lock serves both.


28. The Surprise Catalog

Symptom Cause Fix
PermissionError / Read-only file system writing to ~/.cache, ~/.config Libraries write under $HOME (matplotlib, Hugging Face, numba, pip, Jupyter); readOnlyRootFilesystem: true emptyDir at /tmp; HOME=/tmp or per-library MPLCONFIGDIR, HF_HOME, NUMBA_CACHE_DIR, XDG_CACHE_HOME
tempfile fails under read-only root /tmp is on the read-only layer emptyDir at /tmp (counts toward ephemeral storage, or memory if medium: Memory)
Bus error / No space left on device in PyTorch DataLoader or multiprocessing.shared_memory /dev/shm is 64 MB by default emptyDir: { medium: Memory, sizeLimit: 1Gi } mounted at /dev/shm (charged to the pod's memory)
Every outbound HTTP call is 20–100 ms slower than locally Kubernetes ndots:5: api.example.com is tried against every search domain first; each miss is a DNS round trip on the asyncio default executor Fully qualify (api.example.com.) or dnsConfig.options: [{name: ndots, value: "2"}]; cache DNS in the client
ZoneInfoNotFoundError No system tzdata in the base image tzdata from PyPI as a dependency (zoneinfo falls back to it)
UnicodeEncodeError writing logs/files LANG unset → C locale on old images ENV PYTHONUTF8=1 (UTF-8 mode becomes the default in 3.15, PEP 686)
SSL: CERTIFICATE_VERIFY_FAILED behind a corporate proxy requests/httpx use certifi's bundle, not the OS store you added the CA to truststore.inject_into_ssl(), or SSL_CERT_FILE/REQUESTS_CA_BUNDLE pointing at the system bundle
Wheels fail to install on Alpine or arm64 No musllinux/aarch64 wheel; sdist build needs a compiler Debian-based images; check uv.lock wheel tags for every target platform before choosing a base
Free-threaded 3.13t/3.14t image is slower and bigger Single-thread overhead, per-object locking, extensions without free-threading support re-enable the GIL (with a warning) Only for proven thread-parallel CPU work; check sys._is_gil_enabled() at startup; PYTHON_GIL=0 forces it off. See ../python-mastery/26-free-threading.md
py-spy/perf can't attach in the pod Missing SYS_PTRACE / PERFMON capability; no frame-pointer info Ephemeral debug container with --profile=sysadmin sharing the process namespace; Python 3.12+ -X perf for readable perf stacks. See ../python-mastery/32-profiling.md, ../python-mastery/12-observing-a-process.md
Logs arrive out of order or late Block-buffered stdout PYTHONUNBUFFERED=1 (§13)
Image works on CI, fails locally with Exec format error or wrong-arch .so imports COPY . . copied the developer's host .venv (macOS/arm64 binaries) over the built one .dockerignore with .venv/, __pycache__/, .git/, .env, .pytest_cache/, .mypy_cache/, .ruff_cache/ (§27)
Private index token visible in docker history Credential passed as ARG/ENV --mount=type=secret (§27)
Pod killed during startup on a busy node Startup CPU-throttled under a small limit §21: measure startup under the limit, size the startupProbe from that

29. Cross-References: python-mastery/

This chapter tells you which dial to turn. The python-mastery/ track explains the machine underneath each dial:

Topic in this chapter Where it's explained from first principles
Thread/process sizing, CFS throttling (§21) 06-processes-threads-scheduling.md, 24-the-gil.md
RSS vs PSS vs working set (§22) 07-virtual-memory.md
MALLOC_ARENA_MAX, jemalloc/mimalloc, fragmentation (§13, §22) 08-allocators.md, 35-memory-optimization.md
--preload, COW, gc.freeze(), start methods (§13, §23) 10-signals-fork-exec.md, 15-refcounting-and-ownership.md, 22-garbage-collection.md
PID 1, SIGTERM, graceful shutdown (§24) 10-signals-fork-exec.md
.pyc, optimization levels, code objects (§8, §25) 19-bytecode-and-code-objects.md
Native wheels and what a .so extension is (§12) 17-c-api-and-extensions.md
Free-threaded images (§28) 26-free-threading.md
Measuring cold start and RSS honestly (§20) 31-measurement-methodology.md, 32-profiling.md, 12-observing-a-process.md

30. TL;DR

  • Use uv (not pip, not poetry) for Python dependency management in containers. 10–100× faster, deterministic lockfile, BuildKit-friendly cache.
  • Use uvx for one-shot CLI tools inside the build (ruff, pytest, mypy) — they don't pollute the image.
  • Multi-stage build with python-bookworm-slim (or astral-sh/uv) as builder, distroless as runtime. 70–100 MB images are routine.
  • Cache mounts on /root/.cache/uv with UV_LINK_MODE=copy (avoids the failed-hardlink fallback). Fast local rebuilds; on ephemeral CI, the exported layer cache does the work (§27).
  • UV_COMPILE_BYTECODE=1 + PYTHONDONTWRITEBYTECODE=1 + readOnlyRootFilesystem: true — precompile at build, ban writes at runtime, ~300–500 ms cold start savings.
  • Strip tests, locales, .dist-info cruft, and debug symbols from native .sos — 30–80 MB image savings.
  • Avoid Alpine for Python unless you've audited every wheel's musl availability. Build on Debian slim, run on distroless with the same Python version.
  • .dockerignore the host .venv/, .git/, .env; pass index tokens as BuildKit secrets, never ARG/ENV (§27).
  • MALLOC_ARENA_MAX=2 in multi-threaded apps for 20–40% RSS reduction.
  • PYTHONUNBUFFERED=1, PYTHONHASHSEED=random — always. PYTHONOPTIMIZE only if you compiled .pyc at that level, or it disables them (§25).
  • Builder and runtime must run the same Python minor version. distroless/python3-debian12 is 3.11; ship a uv-managed interpreter on distroless/cc instead.
  • Compile your own code too (--no-editable or compileall), with --invalidation-mode unchecked-hash.
  • os.cpu_count() lies in a pod. Set PYTHON_CPU_COUNT (3.13+), OMP_NUM_THREADS, WEB_CONCURRENCY from the Downward API, or thread pools and BLAS size themselves to the node and get CFS-throttled (§21).
  • OOMKill is SIGKILL, not MemoryError. Watch working set vs memory.max; recycle workers with --max-requests + jitter; consider jemalloc for C-heavy apps (§22).
  • --preload + gc.freeze() to keep copy-on-write pages shared; mind the 3.14 forkserver default (§23).
  • Exec-form entrypoint and a SIGTERM handler (or a server that has one); PID 1 ignores unhandled signals (§24).
  • New projects: uv over Poetry; existing Poetry repos: --only main --no-root dependency layer, Poetry in the builder only, or uvx migrate-to-uv (§26).
  • Lazy-import heavy modules (boto3, pandas, ML libs) from request handlers, not at app startup.
  • One uvicorn per pod, scale via replicas for async services; gunicorn+uvicorn workers with --preload for COW memory savings if you want multi-process per pod.
  • Kubernetes httpGet/grpc probes need nothing in the image; only Docker HEALTHCHECK/exec probes need a tiny Go probe or a python -c one-liner (no curl in distroless).
  • Commit uv.lock. Use --locked in CI. Fail builds if lockfile is stale.
  • Measure before claiming improvement. Image size with dive, cold start by timing pod-ready, memory by memray or cgroup files.

The Python-in-Docker default in 2026 is uv + distroless + multi-stage + bytecode precompilation. Anything else is leaving 50–90% of the performance and 50–80% of the image size on the table.