OpenInfer docs
platform.openinfer.io →

Quickstart

Install the CLI, pair a GPU, load a model, and make your first API call.

Go from nothing to your own OpenAI-compatible inference endpoint, running on a GPU you control. Five steps:

  1. Create an account
  2. Install the weave CLI and sign in
  3. Pair a GPU node
  4. Load a model
  5. Make your first API call

You bring the GPU (a cloud instance or your own hardware); OpenInfer's hub routes requests to it. There is no separate "host" vs "worker" — every node is symmetric.

Set up with an AI agent

Let a coding agent (Claude Code, Codex, Cursor) do the steps below for you. Paste this prompt into it:

Set up Loom for me: https://platform.openinfer.io/agent-setup.md

The agent fetches /agent-setup.md, a step-by-step setup prompt written for agents, and follows it from installing the CLI to making the first API call. It stops once, for the browser sign-in. The rest of this page covers the same steps by hand.

Just want a hosted API and no GPU? Skip steps 3–4: create an account, mint a pk_… key (step 5), and call any model already served by the shared @oi/… catalog. Steps 3–4 are only for serving models on your own hardware.

1. Create an account

Sign up at https://platform.openinfer.io with Google or an email address. This creates your user and your organization. OpenInfer is in invite-only beta: accept the invitation you were sent to get in. Without one you land on a "Coming Soon" page.

An organization owns your nodes, models, and API keys. You can belong to more than one; CLI commands act on one organization at a time.

2. Install the weave CLI and sign in

weave is a single public binary. Install it on your workstation — no token, Docker, or GPU drivers required:

curl -fsSL https://platform.openinfer.io/install.sh | sh

Windows (PowerShell):

irm https://platform.openinfer.io/install.ps1 | iex

The default install lays down only the CLI. The inference runtime (oi-runtime) is installed later, on the GPU box, when you pair it.

Sign in with the browser device flow:

weave auth login          # prints a URL; approve it in your browser

weave whoami              # confirm the signed-in organization

If you belong to multiple organizations:

weave orgs ls

weave orgs use <organization-id>

3. Pair a GPU node

You need a machine with a GPU (for example an AWS g6e/g7e instance, or your own box). This guide assumes it exists and you can run commands on it. The node must be Linux x86_64 with glibc ≥ 2.34 (Ubuntu 22.04-class or newer) and, for a GPU, an NVIDIA driver ≥ 580 (the runtime is built against CUDA 13). CPU-only works for small models (see the note in step 4). The workstation you run weave from has no such constraint.

Running nodes in Docker or Kubernetes? The same steps work inside a container. See Deploying with containers for a ready-made image and manifests.

a. Mint a pairing token on your workstation (you're logged in, so no dashboard trip):

weave pairing-tokens mint --label first-node

Copy the weave_pt_… value. It is shown only once. Owners and admins can also mint one in Studio under Organization settings → General → Pairing tokens.

b. On the GPU machine, install the public CLI, pull the runtime with it, then bring the node online:

curl -fsSL https://platform.openinfer.io/install.sh | sh   # public CLI only

weave pair <weave_pt_…>        # store the organization's pairing token

weave runtime install          # pull oi-runtime with that token (auto-detects cpu/cuda)

weave up                       # launch the runtime, connect to the hub, and serve

weave up launches and supervises the local oi-runtime inference runtime, connects the box to the hub, and serves — reconnecting forever and restarting the runtime if it exits, safe to re-run. A bare weave up is a complete serving node. For a boot-persistent service (systemd / Scheduled Task), add --service at install time; there the runtime runs as its own unit and the agent runs weave up --no-managed-engine.

c. Back on your workstation, wait for it to show up:

weave nodes               # your node should appear as connected

Note the node's id from this list.

4. Load a model

List the models your organization can serve:

weave models

If it's empty, your organization has no catalog yet. Import a model from the platform catalog, picking one that fits your node (owner or admin only):

weave models available --fits <node-id>

weave models import @oi/Llama-3.2-1B-Instruct

If the model you need isn't listed, ask us to publish it.

Assign one to your node at high priority and wait for it to load (the hub requires at least one high-priority model per node):

weave push <node-id> --high @oi/Llama-3.2-1B-Instruct --wait

Confirm it's loaded and routable:

weave route @oi/Llama-3.2-1B-Instruct    # lists the nodes now serving it

CPU-only node? Not every model runs on CPU — it needs a CPU-executable quant. If weave node <id> shows the model with loadState: error (the runtime reports a failed validating warmup forward), it can't execute on CPU; use a GPU node or a CPU-built quant. On a GPU node any published @oi/… model works.

5. Make your first API call

Inference uses an organization API key (pk_…), separate from your CLI login. Mint one with the CLI:

weave keys mint

It is printed once. You can also mint keys in Studio under Organization settings → API Keys.

The endpoint is OpenAI-compatible — point any OpenAI SDK at https://platform.openinfer.io/v1:

curl https://platform.openinfer.io/v1/chat/completions \

  -H "Authorization: Bearer pk_…" \

  -H "Content-Type: application/json" \

  -d '{

    "model": "@oi/Llama-3.2-1B-Instruct",

    "messages": [{"role": "user", "content": "Say hello in five words."}]

  }'

Python (OpenAI SDK):

from openai import OpenAI



client = OpenAI(

    base_url="https://platform.openinfer.io/v1",

    api_key="pk_…",

)

resp = client.chat.completions.create(

    model="@oi/Llama-3.2-1B-Instruct",

    messages=[{"role": "user", "content": "Say hello in five words."}],

)

print(resp.choices[0].message.content)

A 200 with a completion means the full path works. A 503 weave_upstream_unavailable means no node currently has that model loaded and ready — re-check step 4 (weave route <model>).

Where to next

  • Deploying with containers: run nodes with Docker, Docker Compose, or Kubernetes instead of a hand-configured box.
  • The full Command reference, covering every weave command and flag.
  • Manage nodes on the fly: weave nodes, weave logs <id> -f, weave restart <id>, weave upgrade <id>.
  • Full command list at any time: weave help.