Skip to main content

Installation

The standard install.sh installer automatically detects the Jetson platform by checking uname -m for aarch64 and the presence of /etc/nv_tegra_release. When a Jetson is detected it will:

  • Generate a Jetson-aware docker-compose.yml (with runtime: nvidia, memory limits, and Jetson-appropriate environment variables)
  • Build the inference image locally from inference/Dockerfile.jetson
  • Pull the remaining images (backend, UI, rag-db) from the private registry
# 1. Download the installer
curl -fsSL https://your-vendor-url/install.sh -o install.sh

# 2. Make it executable and run
chmod +x install.sh
./install.sh

Follow the prompts to enter your registry credentials and license key. The installer handles everything else automatically.

Manual Setup​

Use the manual setup if you need more control over the configuration or if the one-step installer does not meet your needs.

Step 1 — Build the Jetson Inference Image​

The Jetson inference image must be built locally because it is based on NVIDIA's L4T base image for aarch64:

cd inference
docker build -f Dockerfile.jetson -t docbot-inference:jetson .

This step requires an internet connection to pull the L4T base image and install Python packages. The build may take 10–20 minutes on the first run.

Step 2 — Configure the Environment​

Copy the example environment file and fill in your credentials and settings:

cp .env.example .env

Edit .env with your registry credentials, license key, and model path. See the Configuration page for a full reference of available variables.

Step 3 — Start the Stack with the Jetson Compose Override​

Use the Jetson-specific Compose override file to start all services with the correct GPU runtime and memory limits:

docker compose -f docker-compose.yml -f docker-compose.jetson.yml up -d

Step 4 — Place the Model File​

Copy your GGUF model file to the path specified by MODEL_PATH in your .env file (default: ./models/):

mkdir -p models
cp /path/to/your-model.gguf models/

See Model Selection & Sizing for guidance on choosing a model that fits within the Jetson's 8 GB shared RAM budget.