Deploying with containers

Run weave nodes with Docker, Docker Compose, Kubernetes, or any container platform.

A weave node is two processes: the weave agent and the oi-runtime inference runtime it launches and supervises. weave up runs both, so a node fits in one container with weave up as its entrypoint. This page builds that image, then runs it with Docker, Docker Compose, and Kubernetes. The same image works on any platform that runs OCI containers with GPU access, such as ECS, Nomad, or Cloud Run with GPUs.

What a node container needs

Requirement	Why
Linux x86_64, glibc ≥ 2.34	`oi-runtime` needs glibc 2.34 and libstdc++ from GCC 12 or newer, so use Ubuntu 22.04 or newer. Alpine (musl) will not run it.
GPU passed through	NVIDIA Container Toolkit on the host, and an NVIDIA driver ≥ 580 (the runtime is built against CUDA 13). The image supplies the CUDA userspace libraries.
A pairing token	`WEAVE_PAIR_TOKEN=weave_pt_…`, from a secret. Used on first start to pair.
A persistent volume at `/root/.weave`	Holds the node's identity, the runtime's workflow, and the model weight cache. One volume per node.
Outbound network only	HTTPS to `https://platform.openinfer.io` and gRPC to the hub at `https://platform.openinfer.io:8443`. The node dials out, so it needs no inbound ports, `Service`, or load balancer.

Why the volume matters. On first start, weave up pairs with WEAVE_PAIR_TOKEN and the hub issues a node id, which is saved to /root/.weave/runner.toml. On later starts it reuses that identity and its cached weights. Without the volume, every new container pairs as a new node and downloads every model again. The volume holds a credential, so treat it as a secret.

Build the image

The weave CLI is a public download. oi-runtime is gated by a pairing token, so pass the token to the build as a BuildKit secret. A secret is available during the RUN step but is never written to an image layer.

# syntax=docker/dockerfile:1
FROM nvidia/cuda:13.0.0-runtime-ubuntu24.04

# The control plane this node pairs with and downloads the runtime from.
ENV WEAVE_ENDPOINT=https://platform.openinfer.io

# libgomp1: the CPU build of oi-runtime (--accel cpu) links GNU OpenMP.
RUN apt-get update \
    && apt-get install -y --no-install-recommends curl ca-certificates libgomp1 \
    && rm -rf /var/lib/apt/lists/*

# Public CLI. As root it installs to /usr/local/lib/weave/bin.
RUN curl -fsSL https://platform.openinfer.io/install.sh | sh

# Gated runtime. --accel cuda because the build has no GPU to detect.
RUN --mount=type=secret,id=weave_token \
    WEAVE_TOKEN="$(cat /run/secrets/weave_token)" \
    /usr/local/lib/weave/bin/weave runtime install --accel cuda

ENTRYPOINT ["/usr/local/lib/weave/bin/weave"]
CMD ["up"]

WEAVE_TOKEN=weave_pt_… docker build \
  --secret id=weave_token,env=WEAVE_TOKEN \
  -t registry.example.com/weave-node:latest .
docker push registry.example.com/weave-node:latest

Replace registry.example.com here and below with your own registry. The image contains the gated runtime, so the registry must be private.

For a CPU-only node, build with --accel cpu and leave out the GPU request in the examples below. CPU nodes suit small models only (see Quickstart step 4).

Docker

docker run -d --name weave-node \
  --gpus all \
  --restart unless-stopped \
  --stop-timeout 30 \
  -e WEAVE_PAIR_TOKEN=weave_pt_… \
  -e WEAVE_NODE_NAME=gpu-box-1 \
  -v weave-state:/root/.weave \
  registry.example.com/weave-node:latest

On SIGTERM, weave stops taking new requests, lets in-flight requests finish for up to 10 seconds, then disconnects from the hub and stops the runtime. --stop-timeout 30 covers that. Docker's default of 10 seconds cuts the drain short.

Check it from your workstation:

weave nodes                    # the node appears as connected

Logs in a container

The agent and the runtime both write to the container's stdout, so read them with docker logs weave-node or kubectl logs weave-node-0. weave logs <node-id> tails journald by default, and containers have no journald. The runtime's structured trace buffer does work from anywhere, because it is read from the runtime itself:

weave logs <node-id> --component core --level warn

Docker Compose

services:
  weave-node:
    image: registry.example.com/weave-node:latest
    restart: unless-stopped
    stop_grace_period: 30s
    environment:
      WEAVE_PAIR_TOKEN: ${WEAVE_PAIR_TOKEN}
      WEAVE_NODE_NAME: gpu-box-1
    volumes:
      - weave-state:/root/.weave
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: all
              capabilities: [gpu]

volumes:
  weave-state:

WEAVE_PAIR_TOKEN=weave_pt_… docker compose up -d

Kubernetes

Use a StatefulSet, not a Deployment. Each replica is its own weave node and needs its own identity. volumeClaimTemplates gives every pod a dedicated volume that survives restarts and rescheduling. A Deployment that shares one volume across replicas would have several pods claiming the same node id.

The cluster needs NVIDIA GPU scheduling: the NVIDIA GPU Operator, or the device plugin on nodes that already have drivers.

apiVersion: v1
kind: Secret
metadata:
  name: weave-pairing
type: Opaque
stringData:
  WEAVE_PAIR_TOKEN: weave_pt_…
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: weave-node
spec:
  serviceName: weave-node
  replicas: 2
  selector:
    matchLabels:
      app: weave-node
  template:
    metadata:
      labels:
        app: weave-node
    spec:
      terminationGracePeriodSeconds: 30
      imagePullSecrets:
        - name: registry-credentials
      containers:
        - name: weave
          image: registry.example.com/weave-node:latest
          args: ["up"]
          env:
            - name: WEAVE_PAIR_TOKEN
              valueFrom:
                secretKeyRef:
                  name: weave-pairing
                  key: WEAVE_PAIR_TOKEN
            # Show each pod under its own name (weave-node-0, weave-node-1, …).
            - name: WEAVE_NODE_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
          resources:
            limits:
              nvidia.com/gpu: 1
          volumeMounts:
            - name: weave-state
              mountPath: /root/.weave
  volumeClaimTemplates:
    - metadata:
        name: weave-state
      spec:
        accessModes: ["ReadWriteOnce"]
        resources:
          requests:
            storage: 200Gi

GPUs per pod. A pod is one node, and the runtime serves on every GPU the pod is given. Request as many as that node should own: nvidia.com/gpu: 8 for a whole 8-GPU machine, 1 to split a machine into single-GPU nodes.
Volume size. Model weights are cached on the volume. Size it for the largest set of models the node will serve, plus headroom.
Scaling. Raising replicas pairs new nodes on their first start. Lowering it disconnects the highest-numbered pods and the hub stops routing to them. Their claims stay, so scaling back up restores the same node ids. Delete a claim to retire that identity for good.
Planned maintenance. Drain first so in-flight work finishes before the pod stops: weave node <node-id> --disable-routing, then delete or reschedule the pod, then --enable-routing once it's back.

Load models

A containerized node is managed like any other. Once it shows up in weave nodes, assign models per node:

weave push <node-id> --high @oi/Llama-3.2-1B-Instruct --wait

Or plan the whole fleet at once:

weave fleet plan --model @oi/Qwen3.5-27B:40            # dry run
weave fleet plan --model @oi/Qwen3.5-27B:40 --apply    # push the layout

A pod that restarts reconnects with the same node id. If it comes back without its models, re-push the organization's layout with weave fleet reapply.

Upgrading

weave upgrade <node-id> swaps binaries inside the running container, and that change is lost when the container is recreated. For containers, make the image the source of truth instead: rebuild it (docker build --pull --no-cache …) to pick up the latest published weave and oi-runtime, push it, and roll the pods. On Kubernetes, roll with kubectl rollout restart statefulset/weave-node when the tag is unchanged. The volume keeps each node's identity and weight cache across the rollout.

For reproducible rollouts, tag images by build date or version, not only latest.

Environment reference

Variable	Purpose
`WEAVE_PAIR_TOKEN`	Pairing token used by `weave up` when no identity exists yet.
`WEAVE_NODE_NAME`	Display name shown in `weave nodes` and Studio.
`WEAVE_ENGINE_URL`	Runtime address. Defaults to `http://127.0.0.1:1337`. When the host is `127.0.0.1`, `localhost`, or `::1`, `weave up` starts and supervises the runtime unless something already answers there. Any other host is treated as a runtime someone else runs.
`WEAVE_ENDPOINT`	Control-plane URL the node pairs with and downloads the runtime from. The image sets it to `https://platform.openinfer.io`; without it, published `weave` builds use production.
`WEAVE_TOKEN`	Pairing token for `weave runtime install` when the box isn't paired yet, as in the image build.

All weave flags and their environment variables are in the Command reference.