Python Containers with uv/uvx: High-Performance Images, Minimal Memory, Minimal Start Time¶
Python is the worst language to containerize well and the most common language to containerize badly. The defaults are user-hostile: python:3.12 is 1 GB before you write a line of code; pip install is single-threaded and re-resolves the world; requirements.txt is unhashed; native wheels disagree with Alpine's musl; import pandas parses ~2,000 files before main() runs. Every layer of the stack — interpreter, packager, image, runtime — has a default that prioritizes "works for everybody" over "small, fast, secure."
This chapter is about flipping every one of those defaults. The thesis: with uv as the package manager, distroless or python:3.12-slim as the base, multi-stage builds, AOT-compiled .pyc, and tuned interpreter flags, a Python service can be packaged into a 50–100 MB image that cold-starts in 200–400 ms and uses 30–40% less RSS than the naive equivalent. Every dial along the way matters; most teams turn none of them.
If you read chapter 39 (Dockerfile best practices), this chapter is the language-specific deep dive that ties it to Python's particular pathologies. It assumes you're already comfortable with multi-stage builds, BuildKit cache mounts, and distroless bases — we'll go past those, into Python-specific territory. Sections 21–28 cover the advanced, surprising parts: what os.cpu_count() and the memory APIs see inside a cgroup, fork/COW and the GC, PID 1 signals, bytecode invalidation modes, Poetry vs uv, and build-cache deep cuts. For the machine underneath each dial, see the python-mastery/ track (§29 maps the two).
Table of Contents¶
- Why Python Defaults Are Hostile to Containers
uvanduvx: What They Are and Why They Belong in Your Dockerfile- The
uvMental Model: Lockfile, Resolver, Cache - Base Image Choice for Python: A Reality Check
- The Canonical Multi-Stage
uvDockerfile - Cache Mounts for
uv: The Real Win uv syncvsuv pip install: Project Mode vs Pip Compat- Bytecode Compilation:
--compile-bytecodeand Why It Matters - Image Size: Where the Bytes Actually Go in Python Images
- Distroless Python: The Production Default
uvxInside the Build: Tools Without Polluting the Image- Native Dependencies:
psycopg,cryptography,numpy,lxml - Memory: Interpreter Tuning, Allocator Tuning, Per-Worker Sizing
- Cold Start: The Import-Time Tax and How to Pay Less
- ASGI/WSGI Server Choice: uvicorn, gunicorn, granian, hypercorn
__pycache__Strategies and Read-Only Filesystems- Reproducibility with
uv lock - CI/CD Patterns for
uvBuilds - The Gold-Standard Dockerfile, Fully Annotated
- Measuring: How to Tell If You Actually Improved Anything
- CPU: What Python Thinks It Has vs What the cgroup Gives It
- Memory: Limits Python Can't See, OOMKills It Can't Catch
- Fork, Copy-on-Write, and the Garbage Collector
- PID 1, Signals, and Graceful Shutdown
- Bytecode Deep Cuts: Optimization Levels, Invalidation Modes, Version Tags
- Poetry vs uv (vs pip) in Containers
- Build Cache Deep Cuts
- The Surprise Catalog
- Cross-References:
python-mastery/ - TL;DR
1. Why Python Defaults Are Hostile to Containers¶
Run through the data:
python:3.12(Debian-based, full image): ~380 MB compressed, ~1.0 GB uncompressed.python:3.12-slim: ~45 MB compressed, ~125 MB uncompressed. Drops dev headers, docs, locales.python:3.12-alpine: ~20 MB compressed, ~55 MB uncompressed. musl libc.gcr.io/distroless/python3-debian12: ~25 MB compressed, ~70 MB uncompressed. No shell, no package manager.
Then your dependencies pile on. pip install fastapi sqlalchemy psycopg2-binary adds another ~80 MB (FastAPI alone is small; psycopg2-binary ships a libpq inside the wheel; SQLAlchemy ships ~12 MB). Add pandas and you're +200 MB. Add tensorflow and you're at +2 GB. Add torch with CUDA and you're at +4 GB.
The problems compound:
- Slow
pip install. pip's backtracking resolver is correct but single-threaded, and downloads and installs serially. On a 50-dep project, a clean resolve can take 20–60 seconds. - No content-addressed cache by default. pip's wheel cache is per-user, not content-addressed; reinstalling the same package may or may not hit the cache.
__pycache__written at first import. Cold start pays the.py→.pyccompilation tax every time, unless you precompile.- Import time dominated by module loading.
import pandasexecutes several hundred modules (numpy, dateutil, pytz, its own submodules) before returning control, and parses every one of them too if there is no usable.pyc. - No native standard for lockfiles.
pip freeze,requirements.txt,Pipfile.lock,poetry.lock,pdm.lock,pip-tools— six conventions, none built in. - musl vs glibc wheels split. Many wheels are
manylinux2014only; on Alpine, pip falls back to source builds. Now you needgcc,python3-dev, library headers — image triples in size. - Two-step builds without tooling support. Out of the box, building a wheel-only image (no compilers in production) requires hand-rolled stages.
uv and a tight Dockerfile fix every one of these.
2. uv and uvx: What They Are and Why They Belong in Your Dockerfile¶
uv is a Python package manager and resolver, written in Rust by Astral (the makers of Ruff). In one binary, it replaces:
pip(installing).pip-tools(locking).virtualenv/venv(env creation).pipx(running Python CLIs in isolated envs — that'suvx).- Parts of
poetry/pdm(project management withpyproject.toml+uv.lock). pyenv(managing multiple Python versions).
The performance claims are not marketing — uv is 10–100× faster than pip on real workloads. On a fresh resolve of FastAPI + SQLAlchemy + Pydantic + uvicorn + ~20 deps, pip takes ~25 seconds; uv takes ~0.8 seconds. On warm cache, uv reuses already-extracted wheels via hardlinks and uv sync finishes a 50-dep install in well under a second.
Why this matters in a container:
- CI builds drop 30–90 seconds. At 50 builds/day, that's a meaningful CI cost.
- Cache mounts get utilized better.
uvhas a sane, content-addressed global cache (typically/root/.cache/uv) that BuildKit cache mounts handle cleanly. - Reproducibility is built-in.
uv.lockis hashed, deterministic, and resolved with a PubGrub resolver; hashes are verified on install. - Bytecode compilation is one flag.
uv sync --compile-bytecodeoruv pip install --compile-bytecodeprecompiles.pycat install time. - No interpreter required to bootstrap
uv.uvis a static Rust binary; you don't need a Python in the image to install Python. (uv python install 3.12will fetch a CPython if you don't have one.)
uvx is the run-a-Python-CLI-without-installing-it command:
uvx tools run in an isolated environment that does not contain your project or its dependencies. That's right for linters and formatters, which only read source, and wrong for anything that imports your code: uvx pytest or uvx mypy will fail on import yourapp. Run those from the project environment with uv run pytest (see §11).
In a Dockerfile build stage, uvx lets you run code-formatters, linters, test-runners, and migration tools without polluting the image — the tool runs in an ephemeral venv that lives in uv's cache, never in the final image. Compare to the bad pattern of pip install pytest && pytest && pip uninstall pytest (which leaves layer cruft regardless of the uninstall).
3. The uv Mental Model: Lockfile, Resolver, Cache¶
Three concepts:
pyproject.toml: declares dependencies, Python version constraints, and optional dependency groups.uv.lock: the resolved, hashed lockfile. Committed to git. Cross-platform: contains entries for all platforms you care about.- Cache (
~/.cache/uvor$UV_CACHE_DIR): a content-addressed store of downloaded wheels, extracted archives, built source distributions, and resolved version info. Reused across projects.
Workflow:
uv init # bootstrap pyproject.toml + .venv
uv add fastapi sqlalchemy # add deps; updates pyproject.toml + uv.lock
uv sync # install everything in uv.lock into the venv
uv lock # re-resolve and write uv.lock without installing
uv lock --upgrade-package fastapi # bump a single package
uv run python -m app # run inside the venv without `source`
For Docker, the key insight: uv sync is fully deterministic given pyproject.toml + uv.lock + a Python version. Two identical uv sync invocations on the same lockfile produce identical site-packages directories, byte for byte (modulo .pyc mtimes, which we'll fix in §8 and §17).
uv and --locked: in CI, always use uv sync --locked (or --frozen). --locked errors if uv.lock would have to change to satisfy pyproject.toml; --frozen skips the freshness check entirely and installs the lockfile as-is. Both install exactly what the lockfile says; only --locked also proves the lockfile still matches pyproject.toml, so prefer it in CI.
Three flags worth knowing:
--no-dev: skip thedevdependency group. Production image doesn't needpytest,mypy,ruff.--no-install-project: don't install the project itself (yet). We'll copy source and install in a later step to get cache layering right.--locked: fail if lockfile is stale.
4. Base Image Choice for Python: A Reality Check¶
The honest comparison for production:
| Base | Size | Wheels | Shell | Verdict |
|---|---|---|---|---|
python:3.12 |
~1 GB | Both | Yes | Never for runtime. Sometimes for build stage. |
python:3.12-slim-bookworm |
~45 MB | glibc | Yes | Default for build stage. Reasonable for runtime if you need a shell. |
python:3.12-alpine |
~20 MB | musl (often missing) | Yes | Avoid unless you've audited every wheel. |
gcr.io/distroless/python3-debian12 |
~25 MB | glibc | No | Production runtime default, but it is Debian's Python 3.11: the builder must match (§25). |
gcr.io/distroless/cc-debian12 + uv-managed Python |
~40 MB | glibc | No | Distroless with the interpreter version you choose. Used in §19. |
cgr.dev/chainguard/python:latest |
~30 MB | glibc | No | Chainguard's distroless equivalent. Aggressively CVE-tracked. |
ghcr.io/astral-sh/uv:python3.12-bookworm-slim |
~85 MB | glibc | Yes | Bundles uv + Python; convenient for build stages. |
The Alpine warnings are not theoretical. Specific failures you'll encounter:
- Wheel coverage lags on musl. Recent
psycopg2-binary,numpyandcryptographydo publishmusllinuxwheels, but older pins, smaller packages, and new Python versions often don't. Any gap falls back to a source build →gcc, headers, and minutes per build. numpy,pandas,scipy,scikit-learnshipmanylinuxwheels first;musllinuxwheels are newer and sometimes incomplete for older versions.cryptographyhas musl wheels but requiresrustcto build older versions.- DNS behaves differently (single-label, parallel queries — see ch 39 §3).
- Threading stack sizes are smaller; deep recursion in Python (e.g., parsing nested JSON) can crash.
The two-line rule: build in python:3.12-slim-bookworm (or the astral-sh/uv image); ship in distroless :nonroot, with the same Python minor version in both stages. Everything else is a deviation that should be justified.
When not to use distroless:
- Your app shells out to subprocesses (calls
subprocess.run(['ffmpeg', ...])). Distroless has no ffmpeg, no shell. Either bundle the binary you need into the image (copy it from a separate stage) or usepython:3.12-slimfor runtime. - You need to
kubectl execfor ad-hoc debugging. Usekubectl debugwith an ephemeral debug container instead; it's the right tool. - Your app uses
os.systemor anyshell=Truesubprocess call. Refactor that — but if you can't, you need a shell in the image.
5. The Canonical Multi-Stage uv Dockerfile¶
The starting point. We'll improve on this in later sections, but get this in your head as the baseline:
# syntax=docker/dockerfile:1.7
# ----- builder ----------------------------------------------------------
FROM ghcr.io/astral-sh/uv:python3.12-bookworm-slim AS builder
ENV UV_COMPILE_BYTECODE=1 \
UV_LINK_MODE=copy \
UV_PYTHON_DOWNLOADS=never \
PYTHONDONTWRITEBYTECODE=1 \
PYTHONUNBUFFERED=1
WORKDIR /app
# Install deps first (cache-friendly)
COPY pyproject.toml uv.lock ./
RUN --mount=type=cache,target=/root/.cache/uv \
uv sync --locked --no-dev --no-install-project
# Now copy the project and install it as a regular (non-editable) package
COPY src ./src
COPY README.md ./
RUN --mount=type=cache,target=/root/.cache/uv \
uv sync --locked --no-dev --no-editable
# ----- runtime ----------------------------------------------------------
FROM gcr.io/distroless/python3-debian12:nonroot
ENV PYTHONDONTWRITEBYTECODE=1 \
PYTHONUNBUFFERED=1 \
PATH="/app/.venv/bin:${PATH}"
WORKDIR /app
# Copy the built venv from the builder; the project lives inside it
COPY --from=builder --chown=nonroot:nonroot /app/.venv /app/.venv
USER nonroot
EXPOSE 8000
ENTRYPOINT ["python", "-m", "app.main"]
The project uses the standard src layout, with a real package name and a build backend so uv sync can install it:
pyproject.toml # [project] name = "app" ... and:
# [build-system]
# requires = ["uv_build>=0.8,<0.12"]
# build-backend = "uv_build"
uv.lock
src/app/__init__.py
src/app/main.py # app = FastAPI()
tests/test_main.py # from app.main import app
Without a [build-system] table, uv treats the project as a virtual project and never installs it, so import app only works when the working directory happens to be on sys.path.
Version trap.
distroless/python3-debian12ships Debian's Python 3.11. With a 3.12 builder, the venv'spythonsymlink points at/usr/local/bin/python3.12, which doesn't exist in the runtime image, and thecpython-312.pycs would be ignored anyway. Either match the builder to the runtime (ghcr.io/astral-sh/uv:python3.11-bookworm-slim) or ship your own interpreter as §19 does. See §25.
What's happening, line by line:
# syntax=docker/dockerfile:1.7: pin BuildKit frontend.FROM ghcr.io/astral-sh/uv:python3.12-bookworm-slim AS builder: starts from Astral's image which has bothuvand Python 3.12 already, on Debian slim. Saves anapt installstep.UV_COMPILE_BYTECODE=1: tellsuvto compile.py→.pycat install time. (Equivalent to passing--compile-bytecode.)UV_LINK_MODE=copy: forcesuvto copy wheel contents into the venv rather than hardlink. Hardlinks are faster but impossible when the cache mount is on a different filesystem than the project — which it always is in BuildKit. Without this, uv tries, fails, and falls back to copying with aFailed to hardlink fileswarning on every build; setting it makes the intent explicit and the logs clean.UV_PYTHON_DOWNLOADS=never:uvwon't try to download a Python interpreter; it uses the one in the image. Saves bytes and avoids surprises.PYTHONDONTWRITEBYTECODE=1: don't write.pycat runtime (we already did it at build time).PYTHONUNBUFFERED=1: flush stdout/stderr after every write. Critical for container logging: without this, your logs are buffered until the process exits or fills the buffer.COPY pyproject.toml uv.lockbefore source: cache layer for dependencies.--mount=type=cache,target=/root/.cache/uv: BuildKit cache mount onuv's global cache. Massive speedup across builds.uv sync --locked --no-dev --no-install-project: install dependencies but not the project itself yet.COPY srcthen seconduv sync --no-editable: the project is built and installed into.venv/lib/python3.12/site-packages/app/like any dependency, soUV_COMPILE_BYTECODEcompiles it too, and the runtime stage needs only.venv. (The default, editable, would leave a.pthpointing at/app/src: the source would have to be copied separately and would never be precompiled; see §25.)- Final stage: distroless Python, copies the prebuilt
.venv(which now contains the app). Nouv, nopip, no shell in the runtime image. USER nonroot: distroless ships anonrootuser at UID 65532. Use it.
Resulting image: ~75–90 MB compressed for a typical FastAPI/SQLAlchemy service. Compare to ~450–550 MB compressed (over 1 GB uncompressed) for a python:3.12 + pip install build of the same code.
6. Cache Mounts for uv: The Real Win¶
uv's cache directory is the secret sauce. By default, on Linux it's ~/.cache/uv (or $UV_CACHE_DIR). It contains:
- Wheel cache: downloaded
.whlfiles, content-addressed. - Extracted wheel cache: pre-extracted wheel contents (so install is just copy/hardlink).
- Source distribution cache:
.tar.gzsdists. - Built wheels: if
uvhad to build a wheel from an sdist (because no binary wheel was available), it caches the built wheel. - Resolver cache: cached metadata for already-queried packages.
A BuildKit cache mount keeps this directory across builds:
Cache hit rates in practice:
- Same project, same lockfile, different code change: ~100% hit. The
uv syncline completes in <1 second. - Same project, dep version bump: ~95% hit (only the changed package is re-downloaded).
- Different project, overlapping deps: significant hit (this is the global-cache magic —
numpy==1.26.4is in the cache regardless of which project pulled it).
Two important details:
1. Set UV_LINK_MODE=copy with cache mounts.
uv's default link mode is hardlink on Linux: when installing a wheel into a venv, it creates hardlinks from the cache to the venv. Hardlinks are fast and disk-efficient — but they require the source and destination to be on the same filesystem. In BuildKit, the cache mount is a separate filesystem from the build's working directory, so hardlinking fails with EXDEV and uv falls back to copying, with a warning.
Set UV_LINK_MODE=copy (or pass --link-mode=copy) to copy from the start: same result, no failed attempt, no warning. Copying is also what you want anyway, since the final image must contain real files, not links into a cache that won't exist at runtime. Net result: still much faster than pip.
2. Cache mount sharing across stages.
If you have multiple stages doing uv sync (e.g., one for dev deps to run tests, another for production deps), they share the cache mount. No duplicate downloads:
FROM ... AS test
RUN --mount=type=cache,target=/root/.cache/uv \
uv sync --locked # includes dev deps
RUN uv run pytest
FROM ... AS prod
RUN --mount=type=cache,target=/root/.cache/uv \
uv sync --locked --no-dev # production only
Both stages share /root/.cache/uv. The test stage downloads pytest; the prod stage doesn't install it, but numpy (used by both) was downloaded once.
7. uv sync vs uv pip install: Project Mode vs Pip Compat¶
uv has two main install commands:
uv sync: project mode. Readspyproject.tomlanduv.lock. Manages a single venv (typically.venv). The canonical command for project workflows.uv pip install: pip-compatible mode. Readsrequirements.txt. Installs into the current Python environment. Useful for legacy projects or when you don't want apyproject.toml.
In a new project, use uv sync. In a legacy project with requirements.txt, you have two options:
Option A: migrate to pyproject.toml + uv.lock.
uv init --no-readme --no-pin-python
# Hand-edit pyproject.toml to add dependencies, or:
uv add -r requirements.txt # handles comments, markers, and extras
uv lock
Option B: keep requirements.txt, use uv pip install.
COPY requirements.txt ./
RUN --mount=type=cache,target=/root/.cache/uv \
uv pip install --system --no-deps -r requirements.txt
--system installs into the system Python (no venv). --no-deps is dangerous unless you trust your requirements.txt to be fully resolved (use uv pip compile requirements.in -o requirements.txt --generate-hashes to produce a hashed, fully-resolved file).
For new projects, always use uv sync with pyproject.toml + uv.lock. Lockfile-driven workflows are the future; requirements.txt is legacy.
Bonus tip — uv pip compile:
This is the pip-tools replacement. Even if you keep requirements.txt, generating it from requirements.in with uv pip compile --generate-hashes gives you a hashed, fully-pinned, reproducible install file. The Dockerfile can then uv pip install --require-hashes -r requirements.txt for hash-verified installs.
8. Bytecode Compilation: --compile-bytecode and Why It Matters¶
When Python imports a module, it:
- Looks for a
.pycfile (compiled bytecode) in__pycache__/. - If found and the source's mtime/hash matches: load the
.pyc. - If not found or stale: parse the
.py, compile to bytecode, write.pyc, then execute.
On a cold container with a read-only filesystem, step 3 fails to write but still does the parse-and-compile work — every time the process starts. For a service that imports a couple thousand modules at startup (Django + DRF + a few apps; FastAPI + SQLAlchemy + Alembic; anything ML-flavored), compilation can take 300–800 ms at every boot.
uv sync --compile-bytecode (or UV_COMPILE_BYTECODE=1) compiles .pyc at install time, in the build stage. The .pyc ships in the image. Cold start now skips compilation entirely:
# Without --compile-bytecode
container start (350ms)
+ import django (220ms parse + 80ms execute)
+ import drf (180ms parse + 50ms execute)
+ ... = ~700ms before first request
# With --compile-bytecode
container start (350ms)
+ import django (45ms load .pyc + 80ms execute)
+ import drf (30ms load .pyc + 50ms execute)
+ ... = ~250ms before first request
Real numbers from a FastAPI service: cold start drops from ~1.2 s to ~0.4 s with bytecode compilation. Just flip the flag. It costs some build time (seconds, parallelized) and image size (the .pyc files are roughly as large as the .py sources they sit next to). Both are paid once per build instead of on every pod start. Two traps can silently undo it: running with PYTHONOPTIMIZE, and an editable project install (§25).
A related trick — PYTHONDONTWRITEBYTECODE=1 at runtime — prevents Python from writing new .pyc files at runtime. Combined with read-only root filesystems (readOnlyRootFilesystem: true in K8s), this means:
- Build stage compiles
.pycs into the image. - Runtime has all the
.pycs available. - Runtime can't write new
.pycs (no need to, no permission to). - No write attempts to a read-only FS that would fail loudly.
This is the right configuration. Both flags. Always.
9. Image Size: Where the Bytes Actually Go in Python Images¶
Run dive on a typical Python service image. The breakdown:
| Component | Size | Reducible? |
|---|---|---|
| Base OS (distroless or slim) | 25–45 MB | Choose smaller base |
| Python interpreter | 25–30 MB | Already minimal |
site-packages (dependencies) |
50–200 MB | Yes (next sections) |
| Your code | 0.5–5 MB | Already minimal |
__pycache__ |
10–30 MB | Keep for cold start (see note below on stripping .py) |
The dependency layer is where the savings are. Specific patterns:
Strip tests and docs from wheels.
Many wheels ship their tests/ directory and docs. Examples: numpy ships a tests/ directory (~30 MB), scipy (~50 MB), pandas (~15 MB). These are useless in production. Strip them in the build stage:
RUN find /app/.venv -type d -name tests -prune -exec rm -rf {} + \
&& find /app/.venv -type d -name __pycache__ -prune \
-path '*/tests/*' -exec rm -rf {} + \
&& find /app/.venv -type d -name "*.dist-info" -exec sh -c \
'rm -rf "$1/RECORD" "$1/INSTALLER" "$1/REQUESTED"' _ {} \;
That find typically saves 30–100 MB on data-science-flavored images.
Strip the C extension .so debug info.
Many wheels ship native extensions (.so files) with debug symbols. strip them:
Saves 20–50 MB for ML-heavy images (many manylinux wheels are already stripped; measure). Two cautions: strip comes from binutils, which slim and uv images don't include, so with 2>/dev/null || true the step silently does nothing until you apt-get install binutils in the builder; and the || true also hides real failures, so check the size before and after once.
Sourceless images (shipping .pyc without .py). It's tempting to delete the .py files and keep __pycache__. That doesn't work: __pycache__/*.pyc files are only used when the matching .py exists. Sourceless distribution needs the legacy layout, python -m compileall -b (writes module.pyc next to the source), then deleting the .py. You lose readable tracebacks and save roughly the source size. Rarely worth it, and not an IP-protection mechanism (bytecode decompiles).
Remove __pycache__ for unused Python versions.
If wheels ship .pyc files for multiple Python versions in their __pycache__/ (look for *.cpython-39.pyc, *.cpython-310.pyc, *.cpython-311.pyc, etc.), strip those that don't match your runtime:
# Keep only 3.12 bytecode
RUN find /app/.venv -name "__pycache__" -type d -exec sh -c \
'find "$1" -name "*.cpython-*.pyc" ! -name "*.cpython-312.pyc" -delete' _ {} \;
Not common (most wheels only ship .py, with .pyc compiled at install) but worth knowing.
Drop locale data.
locale/ directories under site-packages are huge for libraries with translations (Django, Babel, polib). If your app is English-only:
This is aggressive — your error messages won't be localized — but it's a 10–40 MB save for some apps.
Beware of pip install --no-deps followed by manual deps.
A common "minimize" pattern is to install only what you import. It works, but: dependency graphs are deep, and you'll get confusing import errors at runtime instead of build time. Use --no-deps only when you have a fully-resolved hashed requirements file.
10. Distroless Python: The Production Default¶
gcr.io/distroless/python3-debian12 includes:
- Debian's CPython interpreter: 3.11 on
debian12(3.13 ondebian13), not the version you pick. If your builder uses a different minor version, the venv won't run on it (§25). The gold-standard Dockerfile in §19 avoids this by shipping a uv-managed interpreter ondistroless/cc. - The CPython standard library.
- glibc, ca-certificates, tzdata, /etc/passwd with a
nonrootuser.
It does NOT include:
- pip, setuptools, wheel (none of them are needed at runtime).
- A shell. No
sh, nobash, nobusybox. - Coreutils, curl, wget.
apt,dpkg.
For a Python service, this is fine — you're running python -m yourapp (or your venv's python). No shell needed.
Variant tags:
:latest— current version, root user.:nonroot— current version, runs as UID 65532.:debug— adds a busybox shell. Useful for ad-hoc debugging in CI; do not ship to production.:debug-nonroot— both.
Use :nonroot in production. It maps to runAsUser: 65532 in your pod spec.
Healthchecks in distroless: there's no curl. On Kubernetes this mostly doesn't matter: httpGet, tcpSocket and grpc probes are executed by the kubelet, not inside the container, so they need nothing in the image and cost no in-container CPU. Prefer them:
readinessProbe:
httpGet: { path: /healthz/ready, port: 8000 }
periodSeconds: 5
livenessProbe:
httpGet: { path: /healthz, port: 8000 }
periodSeconds: 10
The curl problem is real for Docker HEALTHCHECK (Compose, ECS) and for exec probes. Two options there:
1. Use a Python-based healthcheck.
livenessProbe:
exec:
command: ["python", "-c", "import urllib.request,sys; urllib.request.urlopen('http://127.0.0.1:8000/healthz', timeout=2)"]
periodSeconds: 10
Adds ~50–80 ms per probe (interpreter startup). Fine for liveness; might be slow for readiness if you probe aggressively.
2. Bundle a tiny health probe binary.
Compile a 5-line Go binary that does an HTTP GET and exits 0/1. Copy it into the image. Distroless still works, probe takes <5 ms.
grpc_health_probe does this for gRPC (on Kubernetes 1.24+, the native grpc: probe replaces it).
11. uvx Inside the Build: Tools Without Polluting the Image¶
uvx (alias for uv tool run) executes a Python CLI in an ephemeral venv, similar to pipx run. Critically, the tool's installation does not affect the project's venv — it lives in uv's cache.
Use cases inside a Dockerfile:
# Run formatter / linter as a build-time check
RUN --mount=type=cache,target=/root/.cache/uv \
uvx ruff check src/
# Pin tool version explicitly
RUN --mount=type=cache,target=/root/.cache/uv \
uvx ruff@0.6.4 check src/
# Audit the locked dependencies for known CVEs
RUN --mount=type=cache,target=/root/.cache/uv \
uv export --locked --no-dev --format requirements-txt > /tmp/req.txt \
&& uvx pip-audit -r /tmp/req.txt --disable-pip
What not to run with uvx: tests and type checkers. uvx pytest runs pytest in an environment without your project or its dependencies, so collection fails on the first import. uvx mypy can't see your dependencies' types. Those belong to the project environment (uv run, below).
None of these tools end up in the final image because:
uvxinstalls into a separate venv (cache-mounted, ephemeral).- The final stage copies only
/app/.venv, not theuvxcache.
This is much cleaner than the legacy:
The pip uninstall doesn't remove the disk bytes from the layer (image is a stack of immutable layers). uvx sidesteps this entirely.
uv run is the project equivalent.
For tools that are dev dependencies of your project (in the dev group of pyproject.toml):
RUN --mount=type=cache,target=/root/.cache/uv \
uv sync --locked # installs dev deps too
RUN uv run pytest
Do this in a separate test stage (§18) so dev dependencies never reach the runtime image. uvx is for tools that don't import your code; uv run in a test stage is for tools that do.
12. Native Dependencies: psycopg, cryptography, numpy, lxml¶
Python wheels often include C extensions. The wheel format includes prebuilt binaries for specific (Python version, platform, libc) tuples. If a wheel matching your environment exists, install is fast and stable. If not, pip/uv falls back to building from source, which needs compilers.
Common offenders and their fixes:
psycopg2-binary vs psycopg2 vs psycopg¶
psycopg2-binaryships libpq inside the wheel. Easy install, but a duplicated libpq per container (security hygiene problem — when libpq has a CVE, you wait forpsycopg2-binaryto release a new wheel).psycopg2(no-binary) needs system libpq at install time and runtime. Requiresapt-get install libpq-devin build stage andlibpq5in runtime stage. Smaller image, system-managed libpq.psycopg(psycopg3) is the modern replacement;psycopg[binary]is the easy version. New code should use psycopg3.
For most projects: psycopg[binary] is fine. For regulated environments: psycopg linked against system libpq.
cryptography¶
Native wheels exist for glibc and musl on amd64 and arm64. Should "just work" with uv on modern bases.
If you see source builds: you've probably pinned an old version. cryptography>=41 has comprehensive wheels.
numpy, pandas, scipy¶
manylinux wheels are reliable on python:3.12-slim. On Alpine, you need musl wheels (sometimes available) or a source build (slow, needs OpenBLAS/LAPACK dev libraries). Don't put data-science workloads on Alpine.
For ML images, consider:
- Use
python:3.12-slimwith prebuilt wheels. - Skip MKL unless you need it (MKL adds ~500 MB and rarely matters for inference).
- For GPU: use NVIDIA's CUDA base images, install Python + uv on top.
lxml¶
Has glibc and musl wheels for current versions. Older Alpine + older lxml = source build needing libxml2-dev, libxslt-dev, gcc.
Generic pattern for unavoidable source builds¶
If you must build from source, use a multi-stage:
FROM ghcr.io/astral-sh/uv:python3.12-bookworm-slim AS builder
RUN apt-get update && apt-get install -y --no-install-recommends \
gcc g++ libpq-dev libxml2-dev libxslt-dev \
&& rm -rf /var/lib/apt/lists/*
ENV UV_COMPILE_BYTECODE=1 \
UV_LINK_MODE=copy
WORKDIR /app
COPY pyproject.toml uv.lock ./
RUN --mount=type=cache,target=/root/.cache/uv \
uv sync --locked --no-dev --no-install-project
COPY src ./src
RUN --mount=type=cache,target=/root/.cache/uv \
uv sync --locked --no-dev --no-editable
# Runtime: no compilers, only runtime libs.
# Same Debian release and same Python minor version as the builder.
FROM python:3.12-slim-bookworm
RUN apt-get update && apt-get install -y --no-install-recommends \
libpq5 libxml2 libxslt1.1 \
&& rm -rf /var/lib/apt/lists/*
COPY --from=builder /app/.venv /app/.venv
WORKDIR /app
USER 65532
ENV PATH="/app/.venv/bin:${PATH}"
ENTRYPOINT ["python", "-m", "app.main"]
The runtime image gets the runtime library packages, not the -dev packages and compilers. Use ldd /app/.venv/lib/python3.12/site-packages/somepackage/_internals.cpython-312-x86_64-linux-gnu.so to discover which libs each native extension needs.
Why not hand-copy the .so files into distroless? It's a common recipe and it's fragile: libpq.so.5 alone pulls in Kerberos, LDAP, and SASL libraries; libxml2 pulls in ICU and liblzma; hard-coded /usr/lib/x86_64-linux-gnu/ paths break the arm64 build; and nothing tells you when a Debian security update changes the set. If you need distroless here, copy the full ldd closure with a script and test the image by importing every native module in CI (python -c "import psycopg2, lxml.etree").
PyTorch and other CUDA-sized wheels¶
The default torch wheel on Linux pulls in the CUDA runtime (several GB). For CPU-only inference, point uv at PyTorch's CPU index; images drop by 2–4 GB:
[tool.uv.sources]
torch = { index = "pytorch-cpu" }
[[tool.uv.index]]
name = "pytorch-cpu"
url = "https://download.pytorch.org/whl/cpu"
explicit = true
13. Memory: Interpreter Tuning, Allocator Tuning, Per-Worker Sizing¶
Python's memory characteristics in containers:
- Resident set size (RSS) is what the kernel counts toward cgroup limits.
- Python rarely returns memory to the OS, so RSS grows toward the high-water mark and stays there.
- Objects ≤ 512 bytes come from
pymallocarenas; everything larger (and every C extension) goes to glibcmalloc, whose per-thread arenas make multi-threaded apps consume more memory than single-threaded equivalents (§22).
Sizing: how much memory does a Python process need?¶
Rough order-of-magnitude:
- Bare Python 3.12 interpreter: 10–15 MB RSS.
- Plus Django + DRF + a few apps: 50–80 MB.
- Plus FastAPI + SQLAlchemy + uvicorn: 60–90 MB.
- Plus
pandasandnumpyimported: 120–180 MB. - Plus a 10 MB ORM cache warmed up: + actual data size.
Set Kubernetes requests.memory near the post-warmup RSS and limits.memory ~2× that to allow growth. Too-tight limits cause OOMKill on transient spikes (large request bodies, ORM queries returning lots of rows, periodic batch jobs).
PYTHONMALLOC and the allocator¶
Python has multiple memory allocators. The default pymalloc is fast for small allocations. For diagnostic builds, you can use PYTHONMALLOC=malloc to bypass pymalloc and route everything to libc malloc — useful for tracking down memory issues with tools that hook libc malloc (valgrind, address sanitizer). In production, leave it default.
MALLOC_ARENA_MAX for multi-threaded workloads¶
glibc's malloc allocates per-thread arenas. By default, it can use up to 8 × num_cpus arenas. On a 16-core machine, that's 128 arenas per process. Each arena reserves virtual address space and over time can hold significant unused memory.
For containerized Python workloads (especially uvicorn with multiple workers, or any process with many threads):
This caps glibc arenas at 2. RSS savings can be 20–40% on multi-threaded apps. Tradeoff: slightly more lock contention on malloc, rarely measurable for normal Python workloads (the GIL serializes most allocation anyway).
PYTHONHASHSEED and security¶
PYTHONHASHSEED=random (the default in Python 3) randomizes hash seeds per process. Important for security (prevents algorithmic complexity attacks against dicts).
PYTHONHASHSEED=0 disables this. Don't. The performance gain is nil; the security cost is real.
PYTHONUNBUFFERED=1 and logging¶
Without this, stdout is line-buffered when attached to a TTY and block-buffered when not (pipes, files, container log streams). In a container, stdout is piped to the container runtime → block-buffered → print() output appears late, out of order relative to stderr, and is lost entirely on SIGKILL/OOMKill. (stderr, where logging writes by default, has been line-buffered since 3.9, which is why logs often look fine while prints don't.)
Always set PYTHONUNBUFFERED=1. Costs nothing.
PYTHONOPTIMIZE¶
PYTHONOPTIMIZE=1: removesassertstatements and sets__debug__to False. Small startup speedup, smaller.pycfiles.PYTHONOPTIMIZE=2: also removes docstrings. ~5–10% memory savings on doc-heavy libraries, and it breaks anything that reads__doc__at runtime: FastAPI's OpenAPI descriptions, Click/Typer help text, PLY grammars, some plugin systems.
If you rely on assertions in production code, don't use this. And if you set it, you must compile bytecode at the same level: uv --compile-bytecode produces level-0 .pycs only, so PYTHONOPTIMIZE=1 makes the interpreter ignore them and recompile from source on every start (§25). The gold-standard Dockerfile leaves it off.
Forking and copy-on-write with gunicorn --preload¶
If you use gunicorn with --preload, the master process imports all your code once, then forks workers. The workers share memory via copy-on-write (COW) — until they touch a page, which then gets duplicated. For Django/Flask apps, this can save 30–60% RSS across all workers.
The catch: any code that opens connections, file descriptors, or starts threads at import time will share them across workers, which is broken. Defer such initialization to a post_fork hook or worker startup.
14. Cold Start: The Import-Time Tax and How to Pay Less¶
Cold start is dominated by imports. Profile your imports:
The log shows hierarchical time spent in each import. Read it; you'll be surprised what's expensive.
Common offenders (Python 3.12 measurements on a modern Linux box):
import numpy: ~80 ms (imports linalg, fft, polynomial, ma, ctypeslib, etc.).import pandas: ~250 ms (imports numpy + a hundred submodules).import django: ~150 ms (imports settings, ORM, template engine).import requests: ~50 ms (imports urllib3, charset_normalizer, idna, certifi).import boto3: ~300 ms (loads service descriptors for ~300 AWS services).import google.cloud.storage: ~400 ms (similar story).
Reduce imports at startup¶
Lazy import frequent vs rare paths. If only 1% of requests need boto3.client('s3'), defer the import:
def upload_to_s3(blob: bytes, key: str) -> None:
import boto3
s3 = boto3.client("s3")
s3.put_object(...)
Now boto3 is imported once per process (on first call), not at startup. Cold start drops by 300 ms.
Use __getattr__ at the module level (PEP 562) for optional features.
# package/__init__.py
def __getattr__(name):
if name == "HeavyClass":
from .heavy_module import HeavyClass
return HeavyClass
raise AttributeError(name)
from package import HeavyClass works as expected, but import package doesn't pull in heavy_module.
Skip pkg_resources.
pkg_resources (from setuptools) is notoriously slow to import (~200 ms). Any library that does import pkg_resources at module load pays this. The modern replacement is importlib.metadata (stdlib, fast). Audit your deps.
Precompiled .pyc (revisited)¶
Covered in §8 — UV_COMPILE_BYTECODE=1 is a free 300–500 ms cold-start win on import-heavy apps. Always do it.
PYTHONDONTWRITEBYTECODE=1 and the read-only filesystem¶
Once you have precompiled .pycs in the image, you don't want runtime to write more. PYTHONDONTWRITEBYTECODE=1 + readOnlyRootFilesystem: true is the right combo. Bonus: prevents an attacker from writing executable bytecode into the container.
python -S (skip site)¶
Python's site.py runs at startup, scanning site-packages and processing .pth files. For ultra-lean services, python -S -m yourapp skips this. But site.py is also what puts the venv's site-packages on sys.path (via pyvenv.cfg; activation scripts only change PATH), so with -S none of your dependencies import until you set PYTHONPATH=/app/.venv/lib/python3.12/site-packages yourself, and .pth-based editable installs stop working. Saves ~10–30 ms. Only worth it for CLIs started thousands of times; for a long-running service it's noise.
Instrumentation and plugin discovery¶
Auto-instrumentation (opentelemetry-instrument, APM agents) and entry-point plugin discovery (importlib.metadata.entry_points(), pytest-style plugin scanning) import extra packages and scan every installed distribution's metadata at startup. Measure them with -X importtime like any other import; enable only the instrumentations you use.
"Warm up" the process before declaring it ready¶
If your app has expensive first-request paths (loading an ML model, opening DB connection pools, JIT compiling a regex), do them at startup, not at first request. Mark readiness only after warm-up completes. Otherwise the first request after pod start is slow, which gets routed to a probe-failing pod, which marks as not-ready, which causes oscillation under autoscaling.
from contextlib import asynccontextmanager
from fastapi import FastAPI, Response
@asynccontextmanager
async def lifespan(app: FastAPI):
app.state.ready = False
await db.connect()
await load_model()
app.state.ready = True
yield
await db.disconnect()
app = FastAPI(lifespan=lifespan)
@app.get("/healthz/ready")
def ready() -> Response:
return Response(status_code=200 if app.state.ready else 503)
Set Kubernetes readinessProbe to hit /healthz/ready. Pod doesn't get traffic until warm.
Cold start budget¶
For a typical FastAPI service: target < 500 ms from docker run to "first request served." Hitting that:
- ~0 ms image pull if the image is already on the node; otherwise seconds, which is why image size matters for scale-out.
- ~100 ms container setup.
- ~200 ms Python interpreter + imports (with precompiled .pyc).
- ~50 ms app startup (DB connection, warm-up).
Above 500 ms, you're losing autoscaling responsiveness. Above 2 seconds, you're losing user experience during scale events.
15. ASGI/WSGI Server Choice: uvicorn, gunicorn, granian, hypercorn¶
For HTTP/web services in containers, the server choice affects memory, CPU, and concurrency model.
| Server | Sync/Async | Multi-process | Use case |
|---|---|---|---|
gunicorn |
WSGI (sync) | Yes (--workers) |
Django, Flask, classic WSGI apps |
uvicorn |
ASGI (async) | Single process (use --workers carefully) |
FastAPI, Starlette, async Django |
gunicorn + uvicorn_worker.UvicornWorker |
ASGI (async) | Yes (gunicorn manages, uvicorn handles) | FastAPI with several processes per pod |
granian |
ASGI/WSGI/RSGI (Rust-backed) | Yes | Performance-critical Python web apps |
hypercorn |
ASGI | Yes | HTTP/2, HTTP/3 needs |
daphne |
ASGI | Yes | Django Channels (WebSockets) |
The right default for FastAPI on Kubernetes: one uvicorn (or granian) process per pod, scaled by replicas (see below). Use gunicorn with uvicorn workers when you need several processes per pod: --preload memory sharing, or isolation between workers. The worker class moved out of uvicorn: uvicorn.workers is deprecated since uvicorn 0.30, so install the uvicorn-worker package and use uvicorn_worker.UvicornWorker.
ENTRYPOINT ["gunicorn", \
"-k", "uvicorn_worker.UvicornWorker", \
"--workers", "4", \
"--worker-tmp-dir", "/dev/shm", \
"--bind", "0.0.0.0:8000", \
"--access-logfile", "-", \
"--preload", \
"app.main:app"]
Notes:
--worker-tmp-dir /dev/shm: gunicorn writes worker heartbeat files. By default/tmp, which may be on the container's overlay disk;/dev/shm(tmpfs) avoids heartbeat stalls when the disk is slow. WithreadOnlyRootFilesystem: true,/tmpisn't writable unless you mount anemptyDirthere, and gunicorn won't start without this flag or that mount.--workers 4: usually2 * CPU + 1, where CPU means the pod's limit, neveros.cpu_count()(§21). In containers, cap based on memory and request profile. Async workers can handle hundreds of concurrent requests each, so don't over-provision workers.--preload: master imports app, forks workers. COW memory savings (see §13).--access-logfile -: log to stdout.
For pure performance, granian is faster than gunicorn+uvicorn:
ENTRYPOINT ["granian", "--interface", "asgi", "--workers", "4", "--host", "0.0.0.0", "--port", "8000", "app.main:app"]
Granian is Rust-based, handles HTTP at the C/Rust layer, dispatches into Python only for the application code. ~20–40% lower CPU per request on benchmarks.
Single uvicorn vs gunicorn+uvicorn¶
In Kubernetes, a common pattern is one uvicorn per pod, scale by replica count:
- Simpler than gunicorn+uvicorn.
- Each pod is one process; no preload, no master.
- Scale concurrency horizontally (more pods, fewer workers per pod).
- Memory is predictable per pod.
- Plays well with Kubernetes autoscaling.
This is increasingly the default for async services. Use gunicorn+uvicorn when you want process-level isolation within a pod (one bad worker crashes, others survive).
Sync Django¶
Django (even with async views in 4+) is mostly sync. Use gunicorn with sync workers:
gthread (gunicorn's threaded sync worker) handles I/O-bound DB queries with threads. Pure sync workers block on the DB.
16. __pycache__ Strategies and Read-Only Filesystems¶
Two consistent strategies for __pycache__:
Strategy A: Precompile at build, ban writes at runtime.
This is the recommended pattern. Bytecode is precompiled, sits in the image, runtime never writes. Read-only FS prevents an attacker from writing a webshell. Cold start is fast.
Strategy B: Allow writes, write at first import.
# Don't set PYTHONDONTWRITEBYTECODE at all: any non-empty value, even "0", disables writes.
ENV PYTHONPYCACHEPREFIX=/var/cache/pycache
PYTHONPYCACHEPREFIX (3.8+) redirects every .pyc write to a separate tree, so the root filesystem can stay read-only. Don't mount the emptyDir over site-packages: an empty volume there hides every installed package.
Not recommended in production. The first request pays the compilation cost. Subsequent requests (same pod) are fast. New pods start cold.
Strategy A wins. The "compile at runtime" strategy was a workaround for build pipelines that couldn't precompile; with uv, you can.
If you must use Strategy B (e.g., dynamic plugins loaded at runtime), at least don't ship a read-only filesystem.
17. Reproducibility with uv lock¶
uv.lock is hashed and deterministic. Two builds from the same lockfile produce identical site-packages directories.
For bit-identical image digests (advanced, see ch 39 §13):
ENV UV_COMPILE_BYTECODE=0 \
UV_LINK_MODE=copy
RUN --mount=type=cache,target=/root/.cache/uv \
uv sync --locked --no-dev --no-install-project
# Hash-based .pyc: content depends only on the source, not on install-time mtimes.
RUN .venv/bin/python -m compileall -q -j 0 --invalidation-mode unchecked-hash .venv/lib
SOURCE_DATE_EPOCH=$(git log -1 --format=%ct) \
docker buildx build --output type=image,name=app,rewrite-timestamp=true .
Two separate problems. Inside a timestamp-mode .pyc, the header embeds the source file's mtime, which differs per install; hash-based .pycs embed a hash of the source instead, so they're identical across builds, and unchecked-hash also skips the freshness check at import (§25). Around the files, every layer tarball records file mtimes; BuildKit's rewrite-timestamp=true clamps them to SOURCE_DATE_EPOCH. Don't put SOURCE_DATE_EPOCH in an ENV line: it flips py_compile's default to checked-hash, which hashes every source file on every import (§25), and touching the .pyc files afterwards changes neither problem.
Most teams don't need bit-identical images. They need dependency-identical images, which uv.lock + --locked gives you for free.
Lockfile hygiene¶
- Commit
uv.lockto git. Always. - Make sure
.dockerignoredoes not excludeuv.lock(the build needs it). It should exclude.venv/,__pycache__/,.git/,.env, and test caches; see §27. - Run
uv lock --upgradeperiodically (Renovate and Dependabot both understanduv.lock) to absorb security fixes. - In CI, fail fast if
uv.lockis stale:
This catches the bug where a contributor added a dependency to pyproject.toml but forgot to uv lock.
18. CI/CD Patterns for uv Builds¶
GitHub Actions¶
name: Build
on: [push]
jobs:
build:
runs-on: ubuntu-latest
permissions: { contents: read, packages: write }
steps:
- uses: actions/checkout@v4
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Log in to GHCR
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Build and push
uses: docker/build-push-action@v6
with:
context: .
push: true
tags: |
ghcr.io/${{ github.repository }}:${{ github.sha }}
ghcr.io/${{ github.repository }}:latest
cache-from: type=gha
cache-to: type=gha,mode=max
provenance: true
sbom: true
platforms: linux/amd64,linux/arm64
cache-to type=gha,mode=max exports all stages' layers; cache-from type=gha restores them. On hosted runners that layer cache is what makes rebuilds fast: the uv cache mount starts empty on every fresh runner and is not exported (§27). With an unchanged uv.lock, the dependency layer is a cache hit and the build takes ~30 seconds; after a lockfile change, it re-downloads everything. Note also that linux/arm64 on an amd64 runner builds under QEMU emulation, which is 5–20× slower; use a native arm64 runner per platform when builds get long.
Testing in the same Dockerfile¶
FROM ... AS test
RUN --mount=type=cache,target=/root/.cache/uv \
uv sync --locked
COPY tests ./tests
RUN uv run pytest -q --tb=short
FROM ... AS build
# ... production stages
docker buildx build --target test . # CI: tests run
docker buildx build --target build . # Deploy: tests are skipped (separate target)
--target test builds up to the test stage and stops. --target build skips the test stage entirely. CI runs both: tests first (gate), then build (the artifact).
A more concise variant — tests inside the build stage:
FROM ... AS builder
COPY pyproject.toml uv.lock ./
RUN --mount=type=cache,target=/root/.cache/uv \
uv sync --locked
COPY src ./src
COPY tests ./tests
RUN --mount=type=cache,target=/root/.cache/uv \
uv run pytest -q
RUN --mount=type=cache,target=/root/.cache/uv \
uv sync --locked --no-dev # remove dev deps for the final venv
Tests run as part of the build; failures abort the build. uv sync is exact by default, so the last step removes the dev dependencies. Don't then copy the whole /app to the runtime stage: tests/ and pytest's caches are still there. Copy only .venv and src.
19. The Gold-Standard Dockerfile, Fully Annotated¶
Putting everything together for a typical FastAPI service:
# syntax=docker/dockerfile:1.7
# ============================================================================
# Stage 1: Builder
# - Astral's uv image (Debian slim + uv); pin it by digest in real use.
# - Installs a uv-managed CPython into /python, so builder and runtime run
# the exact same interpreter binary (no distroless 3.11-vs-3.12 mismatch).
# - Installs deps with cache mount, compiles bytecode at install time.
# - Strips tests, docs, locales from site-packages.
# ============================================================================
FROM ghcr.io/astral-sh/uv:bookworm-slim AS builder
ENV UV_COMPILE_BYTECODE=1 \
UV_LINK_MODE=copy \
UV_PYTHON_INSTALL_DIR=/python \
UV_PYTHON_PREFERENCE=only-managed \
UV_PYTHON=3.12 \
PYTHONDONTWRITEBYTECODE=1 \
PYTHONUNBUFFERED=1
# binutils only for `strip` below; the builder stage never ships.
RUN apt-get update && apt-get install -y --no-install-recommends binutils \
&& rm -rf /var/lib/apt/lists/*
RUN uv python install 3.12
WORKDIR /app
# Dependency layer
COPY pyproject.toml uv.lock ./
RUN --mount=type=cache,target=/root/.cache/uv \
uv sync --locked --no-dev --no-install-project
# Project layer: non-editable, so the app is installed (and compiled) into the venv
COPY src ./src
COPY README.md ./
RUN --mount=type=cache,target=/root/.cache/uv \
uv sync --locked --no-dev --no-editable
# Slim the venv
RUN find /app/.venv -type d -name tests -prune -exec rm -rf {} + \
&& find /app/.venv -type d -name "*.dist-info" -exec sh -c \
'for d; do rm -f "$d/RECORD" "$d/INSTALLER" "$d/REQUESTED"; done' _ {} \; \
&& find /app/.venv -name "*.so" -exec strip --strip-unneeded {} + 2>/dev/null || true \
&& find /app/.venv -path "*/locale/*" \( -name "*.mo" -o -name "*.po" \) -delete
# Slim the interpreter: no pip, IDLE, or Tk at runtime (~20 MB).
RUN find /python \( -name ensurepip -o -name idlelib -o -name tkinter -o -name turtledemo \
-o -name pip -o -name 'pip-*.dist-info' -o -name 'tcl9*' -o -name 'tk9*' \
-o -name 'libtcl*' -o -name '_tkinter*.so' \) -prune -exec rm -rf {} +
# ============================================================================
# Stage 2: Test (optional, run via --target test)
# ============================================================================
FROM builder AS test
RUN --mount=type=cache,target=/root/.cache/uv \
uv sync --locked --no-editable # bring in dev deps; tests import the installed `app`
COPY tests ./tests
RUN uv run --no-sync pytest -q tests/
# ============================================================================
# Stage 3: Runtime
# - Distroless cc (glibc + libstdc++, no Python), nonroot.
# - Only the interpreter and the built venv (which contains the app).
# - No shell, no package manager.
# ============================================================================
FROM gcr.io/distroless/cc-debian12:nonroot AS runtime
# No PYTHONOPTIMIZE: the .pyc files above are level 0, and -O would ignore them (§25).
ENV PYTHONDONTWRITEBYTECODE=1 \
PYTHONUNBUFFERED=1 \
PYTHONHASHSEED=random \
MALLOC_ARENA_MAX=2 \
PATH="/app/.venv/bin:${PATH}"
WORKDIR /app
COPY --from=builder /python /python
COPY --from=builder --chown=nonroot:nonroot /app/.venv /app/.venv
USER nonroot
EXPOSE 8000
# Bind to 0.0.0.0 inside the pod; service maps the port.
ENTRYPOINT ["python", "-m", "uvicorn", "app.main:app", \
"--host", "0.0.0.0", "--port", "8000", \
"--no-access-log"]
Annotations:
- Cache mounts on the global
uvcache mean repeated builds reuse downloaded wheels. --no-install-projectin the first sync gives a clean dependency layer; project install in the second sync is fast and re-runs only when source changes.uv run --no-syncin the test stage runs pytest against the environment exactly as synced, instead of lettinguv runre-sync (and switch the project back to editable).--lockedensuresuv.lockis the source of truth (build fails if it's stale).- Strip tests/dist-info/locales trims ~30–80 MB.
strip --strip-unneededon.sofiles removes debug info (needsbinutils, installed in the builder only).- Slim the interpreter: the managed CPython is ~120 MB uncompressed (the
python3.12binary is statically linked, ~30 MB, and ships next to a ~30 MBlibpython), about twice Debian's. Deleting pip, IDLE, and Tk recovers ~20 MB. That size is the price of choosing your Python version; don'tstripthe interpreter itself (it's already stripped; ~3 MB gain). UV_PYTHON=3.12pins the interpreter for bothuv python installanduv sync, so arequires-python = ">=3.12"can't make uv download a newer version behind your back.- uv-managed interpreter on
distroless/cc: the venv'spythonsymlink resolves to/python/...in both stages, so the.pycversion tag and the binary always match.python3-debian12would give you Debian's 3.11 instead (§25). --no-editableinstalls the project intosite-packageslike a dependency:UV_COMPILE_BYTECODEcompiles it, the runtime copies one directory, andimport appnever depends on the working directory (§25). Project layout is shown in §5.- Distroless nonroot runtime; no shell, no pip, no curl. Smallest reasonable surface.
MALLOC_ARENA_MAX=2for memory hygiene.- No
PYTHONOPTIMIZE: it would make the interpreter ignore every level-0.pycwe just compiled (§25). PYTHONHASHSEED=randomexplicit (defensive, also the default).- Single uvicorn process per pod, scale via replicas.
--no-access-logfor low CPU overhead; ship access logs from a sidecar if needed.
Expected result for a FastAPI service with ~30 deps:
- Image size: 70–90 MB compressed. (Measured for this exact Dockerfile with only
fastapi+uvicorn: 45 MB compressed, 192 MB uncompressed;import app.mainin ~0.4 s from a read-only root, vs ~1.2 s with-O, which ignores the level-0.pycs.) - Cold start (from
docker runto first response): ~300–500 ms. - RSS under idle: ~80 MB.
- RSS under load: ~150–200 MB.
Compare to the "naive" version (python:3.12 + pip install):
- Image size: ~450–550 MB compressed (over 1 GB uncompressed).
- Cold start: ~1.5–2.5 s.
- RSS under idle: ~120 MB.
- RSS under load: ~200–300 MB.
4–10× smaller image, 3–5× faster cold start, 20–30% lower RSS. Same code.
The matching pod spec¶
The image assumes a pod that is read-only, non-root, and tells Python how much CPU and memory it really has (§21, §22):
apiVersion: apps/v1
kind: Deployment
metadata: { name: api }
spec:
replicas: 3
selector: { matchLabels: { app: api } }
template:
metadata: { labels: { app: api } }
spec:
automountServiceAccountToken: false
securityContext:
runAsNonRoot: true
runAsUser: 65532
seccompProfile: { type: RuntimeDefault }
terminationGracePeriodSeconds: 30
containers:
- name: api
image: ghcr.io/acme/api@sha256:... # pin by digest
ports: [{ containerPort: 8000 }]
env:
- name: PYTHON_CPU_COUNT
valueFrom: { resourceFieldRef: { resource: limits.cpu, divisor: "1" } }
- name: OMP_NUM_THREADS
valueFrom: { resourceFieldRef: { resource: limits.cpu, divisor: "1" } }
- name: MEMORY_LIMIT_BYTES
valueFrom: { resourceFieldRef: { resource: limits.memory } }
resources:
requests: { cpu: 500m, memory: 192Mi }
limits: { cpu: "1", memory: 384Mi }
securityContext:
readOnlyRootFilesystem: true
allowPrivilegeEscalation: false
capabilities: { drop: [ALL] }
startupProbe:
httpGet: { path: /healthz, port: 8000 }
periodSeconds: 1
failureThreshold: 30 # measured under the CPU limit, not on a laptop
readinessProbe:
httpGet: { path: /healthz/ready, port: 8000 }
periodSeconds: 5
livenessProbe:
httpGet: { path: /healthz, port: 8000 }
periodSeconds: 10
lifecycle:
preStop: { sleep: { seconds: 5 } } # let endpoints drain before SIGTERM (§24)
volumeMounts:
- { name: tmp, mountPath: /tmp }
volumes:
- name: tmp
emptyDir: { sizeLimit: 64Mi }
20. Measuring: How to Tell If You Actually Improved Anything¶
Don't optimize without measurement. The relevant numbers:
Image size¶
docker images myapp:latest --format "{{.Size}}"
docker inspect myapp:latest | jq '.[0].Size' # uncompressed
docker image history myapp:latest
dive myapp:latest # layer-by-layer with score
For a quick check that you're not adding bloat: dive shows "wasted space" — files added and later removed, which still occupy bytes.
Cold start¶
--entrypoint is required because the image's ENTRYPOINT would otherwise receive python -c ... as arguments; --read-only reproduces the production filesystem, so missing .pycs show up as slow imports. This roughly captures import time. For real cold start, time from pod creation to Ready, with a readiness probe that hits a real endpoint (without one, the pod is Ready the moment the process starts, and you measure nothing):
# In a test cluster
# pod.yaml: the image's own ENTRYPOINT, plus an httpGet readinessProbe on /healthz/ready
START=$(date +%s.%N)
kubectl apply -f pod.yaml
kubectl wait --for=condition=Ready pod/coldstart --timeout=120s
END=$(date +%s.%N)
echo "Cold start: $(echo "$END - $START" | bc) seconds"
kubectl get events --field-selector involvedObject.name=coldstart # Pulling -> Pulled = pull time
Memory¶
# Inside the pod
cat /sys/fs/cgroup/memory.current # cgroup v2
cat /sys/fs/cgroup/memory/memory.usage_in_bytes # cgroup v1
# From Python
import resource; resource.getrusage(resource.RUSAGE_SELF).ru_maxrss # in KB
Memory profiling tools: memray (the modern choice: tracks native allocations too, low overhead, good flame graphs) and tracemalloc (stdlib, Python allocations only). py-spy is a CPU sampling profiler, not a memory one, but it answers the companion question: where startup time goes (py-spy record -- python -m yourapp). ru_maxrss is the peak RSS (KiB on Linux); for the number the OOM killer uses, read memory.current and memory.peak (cgroup v2).
Import time¶
python -X importtime -c "import app.main" 2> imports.log
# Slowest 20 by cumulative time (columns: self | cumulative | package):
sort -t'|' -k2 -n imports.log | tail -20
# Or visualize the tree:
uvx tuna imports.log
Anything above 50 ms per import deserves investigation.
Continuous tracking¶
In production:
- Image size in CI: fail the build if size grows >20% week-over-week.
- Cold start in observability: tag pod start events with a histogram bucket, alert on regressions.
- RSS as a standard cluster metric: alert when p99 approaches memory limit.
Don't trust "I made it smaller" without numbers. Don't trust "it's faster now" without a probe time histogram.
21. CPU: What Python Thinks It Has vs What the cgroup Gives It¶
The most expensive surprise in containerized Python is a number: os.cpu_count(). On a 64-core node, inside a pod with limits.cpu: 2, it returns 64. A CPU limit is a CFS bandwidth quota (cpu.max), not a set of CPUs; every core stays visible, you just get throttled once the cgroup has burned 2 CPU-seconds per second.
Three different questions, three different answers:
| API | What it reads | Respects cpuset / affinity |
Respects CFS quota (limits.cpu) |
|---|---|---|---|
os.cpu_count() |
CPUs online on the host | No | No |
len(os.sched_getaffinity(0)) |
CPUs this process may be scheduled on | Yes | No |
os.process_cpu_count() (3.13+) |
Same as affinity | Yes | No |
PYTHON_CPU_COUNT=N / -X cpu_count=N (3.13+) |
Whatever you set | Overrides all three above | Overrides all three above |
Affinity only shrinks when the kubelet's static CPU manager policy pins a Guaranteed-QoS pod with integer CPUs to dedicated cores. In every other pod, affinity is the whole node too. CPython deliberately does not parse cgroups (gh-109595): the spec is complex and the answer can change at runtime. So the only reliable answer is the one you tell it.
Who calls it behind your back¶
| Consumer | Default sizing | Knob |
|---|---|---|
ThreadPoolExecutor (incl. asyncio's default executor, so every getaddrinfo and to_thread) |
min(32, cpu + 4) |
max_workers=, or PYTHON_CPU_COUNT |
ProcessPoolExecutor, multiprocessing.Pool |
cpu processes |
max_workers= / processes= |
| gunicorn / uvicorn workers | 1 unless set; the classic formula from gunicorn's docs, 2 × cpu + 1, is usually implemented as multiprocessing.cpu_count() * 2 + 1 |
WEB_CONCURRENCY (both honor it), --workers |
| NumPy/SciPy via OpenBLAS, MKL; scikit-learn | one thread per visible core | OPENBLAS_NUM_THREADS, MKL_NUM_THREADS, OMP_NUM_THREADS |
| PyTorch intra-op threads | physical cores | torch.set_num_threads(), OMP_NUM_THREADS |
| Polars, PyArrow, DuckDB | visible cores | POLARS_MAX_THREADS, pyarrow.set_cpu_count(), SET threads |
| glibc malloc arenas | 8 × cores |
MALLOC_ARENA_MAX (§13) |
The failure mode is not a crash. It is CFS throttling: 64 OpenBLAS threads on a 2-CPU quota burn the whole 100 ms period's budget in ~3 ms, then the cgroup is frozen for the remaining ~97 ms. p50 looks fine, p99 has 100 ms steps in it, and container_cpu_cfs_throttled_periods_total climbs. A ProcessPoolExecutor() with no arguments on that same node forks 64 workers, each with its own copy of your imports, and the pod OOMs instead.
Measured: the quota is invisible, the throttling is not¶
Python 3.13.16, process in a cgroup with cpu.cfs_quota_us = 50000 (cgroup v1; cpu.max = 50000 100000 on v2), i.e. 0.5 CPU, on a 2-core host:
os.cpu_count()= 2,os.process_cpu_count()= 2. The quota is invisible to both.- The quota is real: a CPU-bound loop took 5.6 s wall for 2.8 s of CPU, with
nr_throttled = 56(one throttled 100 ms period for almost every period it ran). - gunicorn with
workers = multiprocessing.cpu_count() * 2 + 1started 5 Uvicorn workers. Each FastAPI worker: ~68 MB RSS / ~49 MB PSS. Total PSS with master: 264 MB. - Extrapolated to a 64-core node: 129 workers × ~49 MB ≈ 6.3 GB, for a pod that was given half a CPU.
Reproduced in Docker (--cpus=0.5, Python 3.13.14, 4-core host, a minimal FastAPI app): cpu_count() = process_cpu_count() = 4; the same loop took 5.6 s wall for 2.8 s CPU with nr_throttled = 57; gunicorn started 9 workers at ~48 MB RSS / ~31 MB PSS each, 300 MB PSS total. Adding --cpuset-cpus=0,1 changed process_cpu_count() to 2 while cpu_count() stayed 4: affinity is visible, quota never is.
Read your own pod's numbers with cat /sys/fs/cgroup/cpu.stat (v2: nr_throttled, throttled_usec) and PSS per worker from /proc/<pid>/smaps_rollup.
Common objections¶
- "gunicorn defaults to 1 worker." True. The problem is the formula from its docs, which many configs implement with
multiprocessing.cpu_count(); that's the code path measured above. - "Use the static CPU manager." It does shrink affinity, so
process_cpu_count()becomes correct, but only for Guaranteed pods with whole-CPU requests on nodes configured for it. Every Burstable pod, and every fractional limit, still sees the whole node. - "500m rounds up to 1." Yes: the Downward API with
divisor: "1"rounds up, so a 500m limit givesPYTHON_CPU_COUNT=1. That's the safe direction: one worker on half a CPU, not zero, not 129. - "We're on Python < 3.13."
PYTHON_CPU_COUNTdoesn't exist there, and is silently ignored. SetWEB_CONCURRENCYand the thread-count variables (OMP_NUM_THREADS,OPENBLAS_NUM_THREADS,MKL_NUM_THREADS,POLARS_MAX_THREADS) explicitly; they work on every Python version. For pools you create in code, size them witheffective_cpus()below.
The fix: tell Python the truth via the Downward API¶
env:
- name: PYTHON_CPU_COUNT # 3.13+: fixes os.cpu_count() and every default derived from it
valueFrom:
resourceFieldRef:
resource: limits.cpu
divisor: "1" # rounds UP: 500m -> 1, 1500m -> 2
- name: OMP_NUM_THREADS
valueFrom:
resourceFieldRef: { resource: limits.cpu, divisor: "1" }
- name: OPENBLAS_NUM_THREADS
valueFrom:
resourceFieldRef: { resource: limits.cpu, divisor: "1" }
- name: WEB_CONCURRENCY
value: "1" # async service: one process per pod, scale with replicas
Two caveats: if the pod has no CPU limit, limits.cpu resolves to the node's allocatable CPU, so you are back to 64; teams that drop CPU limits (a defensible choice, to avoid throttling) must set these to a fixed number instead. And on Python ≤ 3.12 PYTHON_CPU_COUNT is ignored, so the explicit variables above carry the whole fix; for pools sized in code, read the quota yourself:
import math
import os
def effective_cpus() -> int:
"""CPUs this process can actually burn: min(affinity, cgroup quota)."""
n = len(os.sched_getaffinity(0))
try:
with open("/sys/fs/cgroup/cpu.max") as f: # cgroup v2
quota, period = f.read().split()
except OSError:
try: # cgroup v1
with open("/sys/fs/cgroup/cpu/cpu.cfs_quota_us") as f:
quota = f.read().strip()
with open("/sys/fs/cgroup/cpu/cpu.cfs_period_us") as f:
period = f.read().strip()
except OSError:
return n
if quota not in ("max", "-1"):
n = min(n, max(1, math.ceil(int(quota) / int(period))))
return n
Cold start is CPU-bound, and the limit applies to it¶
Imports, .pyc unmarshalling, and warm-up are pure CPU. A service that starts in 400 ms with a full core starts in ~1.6 s at limits.cpu: 250m, and the startupProbe you tuned on a laptop fails. Either give requests/limits headroom for startup, set the startupProbe from a measurement taken under the limit, or (Kubernetes 1.33+, in-place pod resize) start with more CPU and resize down after ready.
Deeper: how the scheduler sees threads and processes, ../python-mastery/06-processes-threads-scheduling.md; why the GIL makes extra threads useless for CPU-bound Python but not for BLAS, ../python-mastery/24-the-gil.md.
22. Memory: Limits Python Can't See, OOMKills It Can't Catch¶
Python sees the host's RAM¶
os.sysconf("SC_PHYS_PAGES") * os.sysconf("SC_PAGE_SIZE") and psutil.virtual_memory().total report the node's memory, not the cgroup's. Anything that sizes a cache "as 25% of RAM" (some ML loaders, in-process LRU caches, a homegrown batcher) sizes it against 256 GB inside a 1 GB pod. Read the limit, or inject it:
def memory_limit_bytes() -> int | None:
try:
with open("/sys/fs/cgroup/memory.max") as f:
v = f.read().strip()
return None if v == "max" else int(v)
except OSError:
return None
MemoryError will not save you¶
When the cgroup hits memory.max, the kernel OOM killer sends SIGKILL. No exception, no traceback, no atexit, no flushed logs; the container exits 137 and kubectl describe says OOMKilled. MemoryError only happens when a single malloc fails, which in a cgroup almost never happens first. If you want a graceful failure, it has to be a self-imposed limit below the cgroup's: reject work when memory.current crosses ~85% of memory.max, or recycle the worker (below).
What the kubelet actually counts¶
memory.currentincludes page cache your process caused (files read, logs written to a file in the container,tmpfs).- The kubelet evicts and the dashboards alert on working set =
memory.current − inactive_file. That is the number to compare against the limit, notpsRSS. - RSS double-counts shared pages across processes. With 4 gunicorn workers sharing a preloaded app, summing RSS over-reports. Use PSS (
/proc/<pid>/smaps_rollup) per process andmemory.currentfor the pod. emptyDir: { medium: Memory }and/dev/shmwrites are RAM charged to the pod.
Fragmentation: RSS that never comes back¶
Python almost never returns memory to the OS, for two independent reasons:
- pymalloc (objects ≤ 512 bytes) carves 1 MiB arenas (3.10+) into 16 KiB pools. An arena is released only when every object in it is freed. One long-lived dict entry pins the megabyte.
- glibc malloc (everything bigger, plus every C extension) only trims the top of the heap; holes in the middle stay resident. Per-thread arenas multiply this (§13).
The levers, cheapest first:
MALLOC_ARENA_MAX=2(already in §13).- Recycle workers:
gunicorn --max-requests 2000 --max-requests-jitter 200. Crude, bounded, works for any leak or fragmentation source. The jitter stops all workers restarting at once. - Swap the allocator for C-extension-heavy apps:
LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libjemalloc.so.2(copy the.sointo a distroless image from the builder). jemalloc's decay returns memory on its own; typical RSS reduction is 10–30% on pandas/numpy-heavy services. Measure; it occasionally costs CPU. CPython's free-threaded build already uses bundled mimalloc. ctypes.CDLL("libc.so.6").malloc_trim(0)after a known large, transient allocation (a batch job, a big report). A targeted hammer, not a default.
Diagnosis ladder and allocator internals: ../python-mastery/08-allocators.md (§5 on arenas, §6 on why memory doesn't come back, §12 on installing jemalloc/mimalloc); RSS vs PSS vs working set: ../python-mastery/07-virtual-memory.md; cutting RSS you can explain: ../python-mastery/35-memory-optimization.md.
Transparent huge pages¶
If the node runs THP in always mode, sparse heaps get backed by 2 MiB pages and RSS jumps for no code change. Most distros default to madvise, which is fine. If you see unexplained RSS growth after a node image upgrade, check /sys/kernel/mm/transparent_hugepage/enabled before blaming the app.
23. Fork, Copy-on-Write, and the Garbage Collector¶
§13 promised that gunicorn --preload shares memory across workers via copy-on-write. In CPython that promise leaks, because reading an object writes to it: every reference taken increments ob_refcnt in the object header, dirtying its page, and the kernel copies the page into the child. A full GC pass is worse: it walks every tracked object and writes GC bookkeeping into each one. A preloaded Django app that shares 80% of its pages right after fork can be down to 20% shared an hour later.
What helps:
- Python 3.12+ immortal objects (PEP 683):
None,True, small ints, interned strings, and many static objects no longer have their refcount written. Free improvement from upgrading. gc.freeze()(3.7+) moves every object currently tracked into a permanent generation the collector never scans, so GC in the children doesn't dirty the parent's pages. The documented recipe is disable early in the parent, freeze right before fork, re-enable in the child:
# gunicorn.conf.py
import gc
preload_app = True
max_requests = 2000
max_requests_jitter = 200
gc.disable() # master: no collections creating holes while importing
def pre_fork(server, worker): # runs in the master, immediately before each fork
gc.freeze()
def post_fork(server, worker): # runs in each worker
gc.enable()
Measure with the sum of Shared_* in /proc/<worker>/smaps_rollup, before and after an hour of traffic. Instagram, who upstreamed gc.freeze(), reported a measurable capacity gain per host from it.
The multiprocessing start method changed under you¶
- Python 3.12 emits a
DeprecationWarningwhen youfork()a process that has threads, because the child inherits locks held by threads that no longer exist (the classic logging/mallocdeadlock). - Python 3.14 changed the default start method on Linux from
forktoforkserver. Code that relied on children inheriting a preloaded model, an open DB pool, or module globals now pays a fresh import per worker, and anything passed to a worker must be picklable. If you depend on fork semantics, say so:multiprocessing.get_context("fork"), and accept the threads caveat.
The fork mechanics, the deadlock reproduced, and the 3.14 change: ../python-mastery/10-signals-fork-exec.md (§8–§10). Why refcounting writes on read: ../python-mastery/15-refcounting-and-ownership.md. The collector walked line by line: ../python-mastery/22-garbage-collection.md.
24. PID 1, Signals, and Graceful Shutdown¶
Whatever ENTRYPOINT starts is PID 1 in the container's PID namespace, and the kernel treats PID 1 specially: a signal whose disposition is the default is dropped, even when sent from outside (only SIGKILL/SIGSTOP are forced). CPython installs a handler for SIGINT but not for SIGTERM. So:
| Entrypoint | What kubectl delete pod does |
|---|---|
python worker.py, no SIGTERM handler |
SIGTERM ignored → nothing happens for terminationGracePeriodSeconds (30 s) → SIGKILL. Every rollout takes 30 s per pod and in-flight work is cut. |
CMD python app.py (shell form) |
/bin/sh -c is PID 1 and doesn't forward signals; same 30 s, and Python never hears about it. In distroless there is no shell, so the container doesn't even start. |
uvicorn / gunicorn / granian (exec form) |
They install SIGTERM handlers: stop accepting, drain, exit. Correct. |
| Anything that spawns subprocesses | PID 1 must also reap zombies. Python only reaps children it wait()s on. |
Rules:
- Always exec form (
ENTRYPOINT ["python", "-m", ...]). - For plain scripts and job runners, handle SIGTERM explicitly:
import signal
import sys
def _terminate(signum, frame):
raise SystemExit(0) # unwinds: finally blocks, context managers, atexit run
signal.signal(signal.SIGTERM, _terminate)
- If the process spawns children it doesn't reap, run under an init:
docker run --init,shareProcessNamespace, or copytiniinto the image andENTRYPOINT ["/tini", "--", "python", ...]. - Draining is not just the signal. Endpoint removal propagates to kube-proxy/ingress after SIGTERM is sent, so a pod that exits instantly still drops requests. Add
lifecycle.preStop: { sleep: { seconds: 5 } }(Kubernetes 1.30+ native sleep; no shell needed) and keep the server's graceful timeout belowterminationGracePeriodSeconds.
How CPython defers handlers to the eval loop (and why a long C call delays your SIGTERM): ../python-mastery/10-signals-fork-exec.md (§3, §4, §13 on reaping, §15 on graceful shutdown assembled).
25. Bytecode Deep Cuts: Optimization Levels, Invalidation Modes, Version Tags¶
§8 says "flip UV_COMPILE_BYTECODE=1 and you're done." Three ways it silently isn't:
Surprise 1: PYTHONOPTIMIZE looks for different files¶
.pyc files are tagged by optimization level:
__pycache__/models.cpython-312.pyc # level 0 (default)
__pycache__/models.cpython-312.opt-1.pyc # -O / PYTHONOPTIMIZE=1
__pycache__/models.cpython-312.opt-2.pyc # -OO / PYTHONOPTIMIZE=2
uv --compile-bytecode (like pip) compiles at level 0 only. Run with PYTHONOPTIMIZE=1 and the interpreter looks for .opt-1.pyc, doesn't find it, and compiles from source. With PYTHONDONTWRITEBYTECODE=1 and a read-only root filesystem it can't cache the result, so every start pays the full compile: you've shipped the .pycs and switched them off. Pick one:
- Run at level 0 (what the gold-standard Dockerfile in §19 now does). The startup win from
-Ois small; asserts are cheap. - Or compile everything at the level you run:
python -m compileall -q -j 0 -o 1 /app/.venv/lib(with a non-editable install the app is in there too; repeat-ofor several levels).
Check it in the built image: python -X importtime -c 'import yourapp' 2>&1 | tail -1 should be identical with and without --read-only.
Surprise 2: your own code isn't compiled¶
uv sync installs your project editable by default: site-packages holds a .pth pointing at /app/src, and --compile-bytecode compiles site-packages, not /app/src. The dependencies start fast; your app's own modules compile on every start. Use uv sync --no-editable (the project becomes a normal installed package, compiled with the rest, and only .venv needs copying to the runtime stage; this is what §5 and §19 do). If you must keep an editable install, compileall the source tree yourself:
Surprise 3: invalidation modes, and SOURCE_DATE_EPOCH¶
How a .pyc proves it is fresh (PEP 552):
| Mode | Check on every import | Cost | Use |
|---|---|---|---|
timestamp (default) |
stat() the .py, compare mtime + size |
one syscall per module | dev |
checked-hash |
read and hash the whole .py |
full source read + SipHash per module | rarely |
unchecked-hash |
nothing | zero | immutable images |
Two traps. Setting SOURCE_DATE_EPOCH (common in reproducible-build pipelines) silently flips py_compile's default to checked-hash, so startup gets slower the moment you make builds reproducible. And anything that rewrites mtimes after compilation (BuildKit's rewrite-timestamp=true, some image-squashing tools, COPY from a stage that touched files) makes timestamp .pycs stale, and they're recompiled at runtime. In an image, sources never change after build, so the correct mode is:
Surprise 4: the version tag must match the runtime interpreter¶
cpython-312 in the filename is the interpreter's magic number. Compile with 3.12 in the builder and run on a 3.11 runtime, and every .pyc is ignored (and the venv's python symlink points at a binary that doesn't exist, so it won't start at all). This is the classic distroless trap: gcr.io/distroless/python3-debian12 ships Debian's Python, which is 3.11, not whatever your builder used. §19 fixes it by shipping a uv-managed interpreter on distroless/cc, so builder and runtime run the same binary by construction.
Smaller dials¶
- Frozen stdlib (3.11+): core startup modules are deep-frozen into the binary; that's why bare
python -c passis ~15 ms. Don't disable it (-X frozen_modules=offis for debugging the stdlib). - Lazy imports in the language: PEP 810 (
lazy import x, accepted for Python 3.15) makes §14's hand-written lazy imports a keyword. Until you're on it, the function-local import and PEP 562__getattr__patterns stand. - zipapps (
shiv,pex): one file, convenient for CLIs, but imports from a zip can't use cached.pycunless the archive includes them, and C extensions must be extracted at first run. For services, a venv in an image is faster.
What a code object and a .pyc actually contain: ../python-mastery/19-bytecode-and-code-objects.md.
26. Poetry vs uv (vs pip) in Containers¶
Poetry is the most common thing teams are migrating from. It's a reasonable tool; the question is what it costs in a Dockerfile.
pip + requirements.txt |
Poetry 2.x | uv | |
|---|---|---|---|
| Lockfile | none native; pip-compile/uv pip compile for a hashed, per-platform file |
poetry.lock, cross-platform, hashed |
uv.lock, cross-platform, hashed |
| Resolve speed (cold, ~100 deps) | n/a | tens of seconds | ~1 s |
| Install speed | serial download+install | parallel, Python-speed | parallel, Rust-speed, 10–100× pip |
| Tool footprint in the builder | already there | Poetry + ~40 deps (~70 MB), must be pinned | one static binary, COPY --from=ghcr.io/astral-sh/uv |
| Manages the Python interpreter | no | no | yes (uv python install) |
| BuildKit cache-mount pattern | --mount=type=cache,target=/root/.cache/pip |
POETRY_CACHE_DIR + mount |
/root/.cache/uv + UV_LINK_MODE=copy |
| Bytecode at install | --compile (default on for pip) |
not by default; compileall after |
--compile-bytecode |
PEP 621 [project] table |
n/a | yes, since 2.0 | yes |
Export to requirements.txt |
n/a | poetry-plugin-export (unbundled since 2.0) |
uv export |
| Plugin ecosystem, publishing workflows | n/a | mature | thinner |
If you're staying on Poetry, do it right¶
FROM python:3.12-slim-bookworm AS builder
ARG POETRY_VERSION=2.1.3
ENV POETRY_NO_INTERACTION=1 \
POETRY_VIRTUALENVS_IN_PROJECT=1 \
POETRY_CACHE_DIR=/tmp/poetry-cache
RUN pip install --no-cache-dir "poetry==${POETRY_VERSION}" # builder only, never in runtime
WORKDIR /app
COPY pyproject.toml poetry.lock ./
RUN --mount=type=cache,target=/tmp/poetry-cache \
poetry install --only main --no-root --no-directory # dependency layer
COPY src ./src
RUN poetry install --only main \
&& .venv/bin/python -m compileall -q -j 0 --invalidation-mode unchecked-hash .venv/lib src
The common mistakes: POETRY_VIRTUALENVS_CREATE=false (installs into the system site-packages alongside Poetry's own dependencies, which then ship in the image and can conflict with yours), installing Poetry in the runtime stage, poetry install without --no-root before the source is copied (busts the dependency layer on every code change), and forgetting poetry check --lock in CI so a stale lock reaches the build.
Recommendation and tradeoff¶
New services: uv. The resolver and installer are an order of magnitude faster, the lockfile is equivalent, and uv manages the interpreter, which removes the builder/runtime version-mismatch class of bug (§25). Existing Poetry repos: the migration is mechanical (uvx migrate-to-uv rewrites pyproject.toml and generates uv.lock); do it when build time or CI minutes hurt. If the team depends on Poetry plugins or its publish flow, keeping Poetry for development and using poetry export → uv pip install --require-hashes in the Dockerfile gets most of the speed without changing developer habits, at the cost of two tools and an export step to keep in sync.
27. Build Cache Deep Cuts¶
§6 covered the uv cache mount. What bites in real pipelines:
- Cache mounts don't travel.
--mount=type=cachelives on the builder's local disk.--cache-to type=registry/type=ghaexport layers, never cache mounts. On ephemeral CI runners the mount is empty on every run; the layer cache (lockfile-first ordering, §5) is what actually saves you there. Use--cache-to type=gha,mode=max(ortype=registry,mode=max) so the intermediate dependency layer is exported, not just the final stage. - Without a mount, the cache ships.
RUN uv syncwith no cache mount writes hundreds of MB of wheels into/root/.cache/uvin that layer. Either mount it or pass--no-cache. Verify withdive. - Concurrent builds on one builder can race on the same cache.
--mount=type=cache,target=/root/.cache/uv,sharing=locked, or give each Python version its ownid=uv-py312. - Non-root builds: if the stage runs as a non-root
USER, the mount is owned by root unless you setuid=/gid=on it, and uv falls back to no cache with a warning you won't read. - Bind-mount inputs instead of
COPYing them when they're only needed for one step; it avoids a layer and keeps the step's cache key exact:
RUN --mount=type=cache,target=/root/.cache/uv \
--mount=type=bind,source=uv.lock,target=uv.lock \
--mount=type=bind,source=pyproject.toml,target=pyproject.toml \
uv sync --locked --no-dev --no-install-project
uv cache prune --cibefore saving a CI-level cache (e.g.actions/cacheon~/.cache/uvfor non-Docker jobs): drops pre-built wheels that are faster to re-download than to restore, keeps the ones built from source.- Private package indexes without leaking the token.
ARG/ENVvalues are stored in the image config anddocker history. Pass credentials as a BuildKit secret, which exists only for thatRUN:
RUN --mount=type=cache,target=/root/.cache/uv \
--mount=type=secret,id=uv_index_token,env=UV_INDEX_INTERNAL_PASSWORD \
UV_INDEX_INTERNAL_USERNAME=__token__ \
uv sync --locked --no-dev --no-install-project
(UV_INDEX_<NAME>_USERNAME / _PASSWORD match an index named internal in [[tool.uv.index]]; the env= form of secret mounts needs Dockerfile syntax 1.10+.)
- .dockerignore is part of the cache key. Without it, COPY . . sends .git/, the developer's .venv/ (with host-platform binaries that overwrite the built ones), __pycache__/, and .env secrets into the build, and any change to them busts the layer cache. Minimum: .venv/ **/__pycache__/ .git/ .env .pytest_cache/ .mypy_cache/ .ruff_cache/ dist/ build/.
- Base images go stale. Pin bases by digest for reproducibility and let Renovate/Dependabot bump the digest, so Debian and CPython security patches arrive as reviewable PRs instead of never. Scan the result in CI (trivy image, grype); see ch 39 for signing and SBOMs.
- Multi-arch: building linux/arm64 under QEMU on an amd64 runner is 5–20× slower and turns any sdist build into minutes. Use native arm64 runners and merge manifests; uv.lock is universal, so the same lock serves both.
28. The Surprise Catalog¶
| Symptom | Cause | Fix |
|---|---|---|
PermissionError / Read-only file system writing to ~/.cache, ~/.config |
Libraries write under $HOME (matplotlib, Hugging Face, numba, pip, Jupyter); readOnlyRootFilesystem: true |
emptyDir at /tmp; HOME=/tmp or per-library MPLCONFIGDIR, HF_HOME, NUMBA_CACHE_DIR, XDG_CACHE_HOME |
tempfile fails under read-only root |
/tmp is on the read-only layer |
emptyDir at /tmp (counts toward ephemeral storage, or memory if medium: Memory) |
Bus error / No space left on device in PyTorch DataLoader or multiprocessing.shared_memory |
/dev/shm is 64 MB by default |
emptyDir: { medium: Memory, sizeLimit: 1Gi } mounted at /dev/shm (charged to the pod's memory) |
| Every outbound HTTP call is 20–100 ms slower than locally | Kubernetes ndots:5: api.example.com is tried against every search domain first; each miss is a DNS round trip on the asyncio default executor |
Fully qualify (api.example.com.) or dnsConfig.options: [{name: ndots, value: "2"}]; cache DNS in the client |
ZoneInfoNotFoundError |
No system tzdata in the base image | tzdata from PyPI as a dependency (zoneinfo falls back to it) |
UnicodeEncodeError writing logs/files |
LANG unset → C locale on old images |
ENV PYTHONUTF8=1 (UTF-8 mode becomes the default in 3.15, PEP 686) |
SSL: CERTIFICATE_VERIFY_FAILED behind a corporate proxy |
requests/httpx use certifi's bundle, not the OS store you added the CA to |
truststore.inject_into_ssl(), or SSL_CERT_FILE/REQUESTS_CA_BUNDLE pointing at the system bundle |
| Wheels fail to install on Alpine or arm64 | No musllinux/aarch64 wheel; sdist build needs a compiler |
Debian-based images; check uv.lock wheel tags for every target platform before choosing a base |
Free-threaded 3.13t/3.14t image is slower and bigger |
Single-thread overhead, per-object locking, extensions without free-threading support re-enable the GIL (with a warning) | Only for proven thread-parallel CPU work; check sys._is_gil_enabled() at startup; PYTHON_GIL=0 forces it off. See ../python-mastery/26-free-threading.md |
py-spy/perf can't attach in the pod |
Missing SYS_PTRACE / PERFMON capability; no frame-pointer info |
Ephemeral debug container with --profile=sysadmin sharing the process namespace; Python 3.12+ -X perf for readable perf stacks. See ../python-mastery/32-profiling.md, ../python-mastery/12-observing-a-process.md |
| Logs arrive out of order or late | Block-buffered stdout | PYTHONUNBUFFERED=1 (§13) |
Image works on CI, fails locally with Exec format error or wrong-arch .so imports |
COPY . . copied the developer's host .venv (macOS/arm64 binaries) over the built one |
.dockerignore with .venv/, __pycache__/, .git/, .env, .pytest_cache/, .mypy_cache/, .ruff_cache/ (§27) |
Private index token visible in docker history |
Credential passed as ARG/ENV |
--mount=type=secret (§27) |
| Pod killed during startup on a busy node | Startup CPU-throttled under a small limit | §21: measure startup under the limit, size the startupProbe from that |
29. Cross-References: python-mastery/¶
This chapter tells you which dial to turn. The python-mastery/ track explains the machine underneath each dial:
| Topic in this chapter | Where it's explained from first principles |
|---|---|
| Thread/process sizing, CFS throttling (§21) | 06-processes-threads-scheduling.md, 24-the-gil.md |
| RSS vs PSS vs working set (§22) | 07-virtual-memory.md |
MALLOC_ARENA_MAX, jemalloc/mimalloc, fragmentation (§13, §22) |
08-allocators.md, 35-memory-optimization.md |
--preload, COW, gc.freeze(), start methods (§13, §23) |
10-signals-fork-exec.md, 15-refcounting-and-ownership.md, 22-garbage-collection.md |
| PID 1, SIGTERM, graceful shutdown (§24) | 10-signals-fork-exec.md |
.pyc, optimization levels, code objects (§8, §25) |
19-bytecode-and-code-objects.md |
Native wheels and what a .so extension is (§12) |
17-c-api-and-extensions.md |
| Free-threaded images (§28) | 26-free-threading.md |
| Measuring cold start and RSS honestly (§20) | 31-measurement-methodology.md, 32-profiling.md, 12-observing-a-process.md |
30. TL;DR¶
- Use
uv(not pip, not poetry) for Python dependency management in containers. 10–100× faster, deterministic lockfile, BuildKit-friendly cache. - Use
uvxfor one-shot CLI tools inside the build (ruff, pytest, mypy) — they don't pollute the image. - Multi-stage build with
python-bookworm-slim(orastral-sh/uv) as builder, distroless as runtime. 70–100 MB images are routine. - Cache mounts on
/root/.cache/uvwithUV_LINK_MODE=copy(avoids the failed-hardlink fallback). Fast local rebuilds; on ephemeral CI, the exported layer cache does the work (§27). UV_COMPILE_BYTECODE=1+PYTHONDONTWRITEBYTECODE=1+readOnlyRootFilesystem: true— precompile at build, ban writes at runtime, ~300–500 ms cold start savings.- Strip tests, locales,
.dist-infocruft, and debug symbols from native.sos — 30–80 MB image savings. - Avoid Alpine for Python unless you've audited every wheel's musl availability. Build on Debian slim, run on distroless with the same Python version.
.dockerignorethe host.venv/,.git/,.env; pass index tokens as BuildKit secrets, neverARG/ENV(§27).MALLOC_ARENA_MAX=2in multi-threaded apps for 20–40% RSS reduction.PYTHONUNBUFFERED=1,PYTHONHASHSEED=random— always.PYTHONOPTIMIZEonly if you compiled.pycat that level, or it disables them (§25).- Builder and runtime must run the same Python minor version.
distroless/python3-debian12is 3.11; ship a uv-managed interpreter ondistroless/ccinstead. - Compile your own code too (
--no-editableorcompileall), with--invalidation-mode unchecked-hash. os.cpu_count()lies in a pod. SetPYTHON_CPU_COUNT(3.13+),OMP_NUM_THREADS,WEB_CONCURRENCYfrom the Downward API, or thread pools and BLAS size themselves to the node and get CFS-throttled (§21).- OOMKill is SIGKILL, not
MemoryError. Watch working set vsmemory.max; recycle workers with--max-requests+ jitter; consider jemalloc for C-heavy apps (§22). --preload+gc.freeze()to keep copy-on-write pages shared; mind the 3.14forkserverdefault (§23).- Exec-form entrypoint and a SIGTERM handler (or a server that has one); PID 1 ignores unhandled signals (§24).
- New projects: uv over Poetry; existing Poetry repos:
--only main --no-rootdependency layer, Poetry in the builder only, oruvx migrate-to-uv(§26). - Lazy-import heavy modules (boto3, pandas, ML libs) from request handlers, not at app startup.
- One uvicorn per pod, scale via replicas for async services; gunicorn+uvicorn workers with
--preloadfor COW memory savings if you want multi-process per pod. - Kubernetes
httpGet/grpcprobes need nothing in the image; only DockerHEALTHCHECK/execprobes need a tiny Go probe or apython -cone-liner (no curl in distroless). - Commit
uv.lock. Use--lockedin CI. Fail builds if lockfile is stale. - Measure before claiming improvement. Image size with
dive, cold start by timing pod-ready, memory bymemrayor cgroup files.
The Python-in-Docker default in 2026 is uv + distroless + multi-stage + bytecode precompilation. Anything else is leaving 50–90% of the performance and 50–80% of the image size on the table.