Configuration

GPU CLI uses a gpu.jsonc file in your project root for configuration. This file is created when you run gpu init.

JSON Schema

For IDE autocomplete, add the schema URL at the top of your gpu.jsonc:

{
  "$schema": "https://gpu-cli.sh/schema/v1/gpu.json"
}

Core Settings

gpu_types

Specify GPU types with optional counts. The system tries GPUs in array order.

{
  "gpu_types": [
    { "type": "RTX 4090" }
  ]
}

Common GPU types:

Consumer: RTX 4090, RTX 4080, RTX 3090
Professional: RTX A6000, RTX A5000, A40
Datacenter: A100 PCIe 80GB, H100 PCIe, H100 SXM

If omitted, GPU CLI auto-selects the best available GPU.

For multi-GPU workloads, specify count per GPU type:

{
  "gpu_types": [
    { "type": "A100", "count": 4 }
  ]
}

Specify multiple fallback GPUs in priority order:

{
  "gpu_types": [
    { "type": "H100", "count": 8 },
    { "type": "A100", "count": 8 }
  ]
}

min_vram

Minimum VRAM in GB. Used for GPU fallback when your preferred GPU isn't available.

{
  "gpu_types": [{ "type": "RTX 4090" }],
  "min_vram": 24
}

If RTX 4090 isn't available, falls back to any GPU with 24GB+ VRAM.

max_price

Maximum hourly price you're willing to pay.

{
  "max_price": 1.50
}

Output Sync

outputs

Patterns for files to sync back from the pod. Uses glob patterns.

{
  "outputs": [
    "outputs/",
    "checkpoints/",
    "*.pt",
    "*.safetensors"
  ]
}

exclude_outputs

Patterns to exclude from output sync.

{
  "exclude_outputs": [
    "*.tmp",
    "*.log"
  ]
}

outputs_enabled

Enable/disable output syncing.

{
  "outputs_enabled": true
}

Environment

environment

Declarative environment specification for pod setup.

{
  "environment": {
    "python": {
      "requirements": "requirements.txt"
    },
    "system": {
      "apt": [
        { "name": "ffmpeg" }
      ]
    }
  }
}

Python packages

{
  "environment": {
    "python": {
      "requirements": "requirements.txt",
      "pip_global": [
        { "name": "torch", "version": "2.1.0" },
        { "name": "transformers" }
      ]
    }
  }
}

System packages

{
  "environment": {
    "system": {
      "apt": [
        { "name": "ffmpeg" },
        { "name": "git-lfs" }
      ]
    }
  }
}

Shell commands

{
  "environment": {
    "shell": {
      "steps": [
        { "run": "pip install -e ." },
        { "run": "chmod +x scripts/setup.sh && ./scripts/setup.sh" }
      ]
    }
  }
}

Downloads

Pre-download models and assets to the pod.

HuggingFace models

{
  "download": [
    {
      "strategy": "hf",
      "source": "black-forest-labs/FLUX.1-dev"
    }
  ]
}

HTTP downloads

{
  "download": [
    {
      "strategy": "http",
      "source": "https://example.com/model.bin",
      "target": "models/model.bin"
    }
  ]
}

Git repositories

Clone tool repositories like ComfyUI with auto-update support:

{
  "download": [
    {
      "strategy": "git",
      "source": "https://github.com/comfyanonymous/ComfyUI",
      "target": "ComfyUI"
    }
  ]
}

Pin to a specific version with branch, tag, or commit:

{
  "download": [
    {
      "strategy": "git",
      "source": "https://github.com/comfyanonymous/ComfyUI",
      "target": "ComfyUI",
      "tag": "v0.3.7"
    }
  ]
}

Options:

branch: Clone specific branch
tag: Checkout specific tag (detached HEAD)
commit: Checkout specific commit hash (detached HEAD)
depth: Clone depth (default: 1 for shallow clone, 0 for full history)

The git strategy auto-pulls on subsequent runs if the working tree is clean. If you've made local modifications (like installing custom nodes), it will warn and preserve your changes.

Civitai models

Download models from Civitai using model IDs or AIR URNs:

{
  "download": [
    {
      "strategy": "civitai",
      "source": "4384"
    }
  ]
}

Supported source formats:

"4384" - Model ID (gets latest version)
"4384:128713" - Model ID with specific version
"urn:air:flux1:checkpoint:civitai:618692@691639" - Full AIR URN
"air:sdxl:lora:civitai:328553@368189" - AIR without urn: prefix

AIR (AI Resource Names) provides a standardized way to reference AI models across platforms.

Pod Settings

keep_alive_minutes

Auto-stop timeout in minutes. Default is 5.

{
  "keep_alive_minutes": 10
}

docker_image

Override the base Docker image.

{
  "docker_image": "runpod/pytorch:2.4.0-py3.11-cuda12.4.1-devel-ubuntu22.04"
}

dockerfile

Path to a Dockerfile for custom builds.

{
  "dockerfile": "Dockerfile"
}

workspace_size_gb

Workspace volume size in GB.

{
  "workspace_size_gb": 50
}

Network Volumes

Network volumes provide persistent storage that survives pod restarts. Essential for large models to avoid re-downloading.

network_volume_id

Attach a specific network volume by ID.

{
  "network_volume_id": "vol_abc123xyz"
}

volume_mode

Strategy for network volume usage. Default is "global".

{
  "volume_mode": "global"
}

Options:

"global" - Use the shared global volume (set via gpu volume set-global)
"dedicated" - Use a project-specific volume (auto-created or specify dedicated_volume_id)
"none" - No network volume (ephemeral storage only)

dedicated_volume_id

When using volume_mode: "dedicated", optionally specify an existing volume ID. If omitted, a project-specific volume is auto-created.

{
  "volume_mode": "dedicated",
  "dedicated_volume_id": "vol_project_xyz"
}

Volume Management

Manage volumes via CLI:

# List all volumes
gpu volume list --detailed

# Create a volume and set as global
gpu volume create --name shared-models --size 500 --set-global

# Check volume usage
gpu volume status

See Commands Reference for all volume commands.

Example Configurations

ML Training

{
  "$schema": "https://gpu-cli.sh/schema/v1/gpu.json",
  "gpu_types": [{ "type": "RTX 4090" }],
  "min_vram": 24,
  "outputs": [
    "checkpoints/",
    "logs/",
    "*.pt"
  ],
  "environment": {
    "python": {
      "requirements": "requirements.txt"
    }
  }
}

Inference Server

{
  "$schema": "https://gpu-cli.sh/schema/v1/gpu.json",
  "gpu_types": [{ "type": "A100 PCIe 80GB" }],
  "keep_alive_minutes": 30,
  "download": [
    {
      "strategy": "hf",
      "source": "meta-llama/Llama-2-7b-chat-hf"
    }
  ]
}

ComfyUI

{
  "$schema": "https://gpu-cli.sh/schema/v1/gpu.json",
  "gpu_types": [{ "type": "RTX 4090" }],
  "outputs": ["outputs/"],
  "download": [
    {
      "strategy": "hf",
      "source": "black-forest-labs/FLUX.1-dev"
    }
  ]
}

Full Schema

See the complete JSON Schema at gpu-cli.sh/schema/v1/gpu.json.

Configuration

On this page