NVIDIA GPU Accelerated Workspaces in Kasm on Harvester

This article provides a comprehensive guide on configuring GPU PCI passthrough on a Harvester hypervisor, installing Nvidia drivers within the Kasm-hosting Virtual Machine, configuring the Nvidia container runtime, and verifying GPU visibility within Kasm Workspaces. We will also showcase ready-to-use AI and ML-ready workspaces from the Kasm AI Registry.

[embed]

While this guide focuses on Harvester, the principles are applicable to other hypervisors like Proxmox, vSphere, cloud environments, or even bare-metal Linux deployments.

Important Considerations Before You Begin:

• Dedicated GPU: When you passthrough a GPU via PCI passthrough to a hypervisor (and subsequently to a VM), that GPU becomes unavailable to the host operating system. It’s recommended to have at least two GPUs in your system if your host OS requires a GPU: one for the host and another dedicated to Harvester for Kasm Workspaces.
• vGPU as an Alternative: An alternative to PCI passthrough is using virtual GPUs (vGPUs). However, this typically requires enterprise-grade Nvidia GPUs and licensed drivers, which is beyond the scope of this article.

Step 1: Configuring GPU PCI Passthrough in Harvester

This section details how to pass a physical Nvidia GPU directly to a Harvester Virtual Machine.

Enable PCI Controller in Harvester:

• Navigate to Advanced → PCI Devices in your Harvester UI.
• If not already enabled, enable the PCI controller. This allows Harvester to manage and passthrough PCI devices.

Create an Ubuntu Server VM Image:

• Go to Images → Create.
• Provide a name for the image.
• Upload an ISO image file or paste a URL from where the image can be download. The recommended image is Ubuntu Noble Server Cloud Image (Ubuntu 24.04 LTS).
• Click Create.

Create a Virtual Machine Network:

• Navigate to Networks → Virtual Machine Networks.
• Click Create.
• Name: Give your network a descriptive name (e.g., kasm-vm-network).
• Type: Choose UntaggedNetwork.
• Cluster Network: Select your management cluster network (e.g., mgmt).
• Click Create.

Create an SSH Key (Recommended):

• Go to Advanced → SSH Keys → Create.
• Name: Give your SSH key a name.
• Public Key: Paste your SSH public key.
• Click Create. This will allow passwordless SSH access to your VM.

Create and Configure the Virtual Machine for Kasm:

• Navigate to Virtual Machines → Create.

Basics Tab:

• Name: Assign a name to your VM (e.g., kasm-gpu-server).
• CPU: Define the number of vCPUs.
• Memory: Allocate sufficient RAM.
• SSH Key: Select the SSH key you created.

Volumes Tab:

• Click Add Volume → Existing.
• Image: Select the Ubuntu Noble Server image you uploaded.
• Size: Set an appropriate disk size (e.g., 100 GB or more, depending on your Kasm image storage needs).

Networks Tab:

• Choose the network you created earlier (e.g., kasm-vm-network)
• Click Add Network.

Advanced Options → PCI Devices Tab:

• In the PCI Devices section, search for your Nvidia GPU. You might see multiple entries related to your GPU (e.g., the GPU itself and its associated audio device).
• Identify and select the GPU and its audio device you wish to dedicate to Kasm.
• Click Enable Passthrough.
• Once passthrough is enabled for the desired devices, attach them to your VM.
• Click Create to provision the Virtual Machine.

Verify PCI Passthrough in Harvester:

• Once the VM is in the “Running” state, select it.
• Check the VM’s annotations or details page. You should see your PCI devices listed as allocated to this VM, confirming the passthrough from Harvester’s perspective.

Step 2: Install Kasm Workspaces on the VM

1. SSH into your VM: Use the SSH key and the IP address assigned to your VM to log in.
2. Install Kasm Workspaces: Follow the official Kasm Workspaces documentation for a single-server installation. Typically, this involves downloading the installation script and running a few commands:

cd /tmp 
curl -O https://kasm-static.s3.amazonaws.com/kasm_release_1.17.0.3bf277.tar.gz 
# Check for the latest version 
tar -xf kasm_release_1.17.0.3bf277.tar.gz 
sudo bash kasm_release/install.sh

Upon completion, the installer will output randomly generated credentials for your Kasm deployment. Store these securely. Log in to your Kasm instance via a web browser using these credentials.

Step 3: Install Nvidia Drivers and Container Toolkit in the VM

Even though the GPU is passed to the VM, you need to install Nvidia drivers and the Nvidia container toolkit within the VM’s operating system (Ubuntu Noble in this case) for Docker containers to utilize the GPU.

Verify GPU Visibility in the VM: Inside your VM, run the following command to check if the OS can see the Nvidia PCI device:

lspci | grep -i nvidia

You should see your Nvidia GPU listed.

Install Nvidia Drivers and Container Toolkit: Kasm Workspaces documentation provides a script for this, especially for recommended distributions like Ubuntu Noble. This script typically performs the following actions:

• Checks for an available Nvidia card.
• Adds the official PPA for graphics drivers on Ubuntu.
• Installs the latest available Nvidia drivers compatible with your GPU.
• Installs the nvidia-container-toolkit.
• Configures Docker to use the nvidia runtime.

Save the following script (or one provided by Kasm documentation) to a file (e.g., gpu_install.sh) on your VM:

#!/bin/bash

# Check for NVIDIA cards
if ! lspci | grep -i nvidia > /dev/null; then
    echo "No NVIDIA GPU detected"
    exit 0
fi

add-apt-repository -y ppa:graphics-drivers/ppa

curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg \
  && curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \
    sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
    sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list

apt update
apt install -y ubuntu-drivers-common

# Run ubuntu-drivers and capture the output
DRIVER_OUTPUT=$(ubuntu-drivers list 2>/dev/null)
# Extract server driver versions using grep and regex
# Pattern looks for nvidia-driver-XXX-server
SERVER_VERSIONS=$(echo "$DRIVER_OUTPUT" | grep -o 'nvidia-driver-[0-9]\+-server' | grep -o '[0-9]\+' | sort -n)
# Check if any server versions were found
if [ -z "$SERVER_VERSIONS" ]; then
    echo "Error: No NVIDIA server driver versions found." >&2
    exit 1
fi
# Find the highest version number
LATEST_VERSION=$(echo "$SERVER_VERSIONS" | tail -n 1)
# Validate that the version is numeric
if ! [[ "$LATEST_VERSION" =~ ^[0-9]+$ ]]; then
    echo "Error: Invalid version number: $LATEST_VERSION" >&2
    exit 2
fi
# Output only the version number
echo "Latest version is: $LATEST_VERSION"
ubuntu-drivers install "nvidia:$LATEST_VERSION-server"
apt install -y "nvidia-utils-$LATEST_VERSION-server"
# Install NVIDIA toolkit + configure for docker
apt-get install -y nvidia-container-toolkit
nvidia-ctk runtime configure --runtime=docker

Make the script executable and run it with sudo privileges:

chmod +x gpu_install.sh 
sudo ./gpu_install.sh

Note: A reboot of the VM might be necessary after the drivers are installed.

Verify Driver Installation: After rebooting, SSH back into your VM and run:

nvidia-smi

This command should output details about your Nvidia GPU, the installed driver version (e.g., 570.133.20), and the CUDA version (e.g., 12.8).

Verify Nvidia Docker Runtime: Execute the following command to list all configured Docker runtimes:

sudo docker info | grep -i runtime

You should see nvidia listed among the runtimes (e.g., Runtimes: io.containerd.runc.v2 nvidia runc).

Step 4: Configure and Verify GPU Support in Kasm Workspaces

Now, let’s configure Kasm to recognize and use the GPU.

Verify GPU Detection in Kasm UI:

• Log in to your Kasm Workspaces admin dashboard.
• Navigate to Infrastructure → Docker Agents.
• Select your agent and scroll down to the GPU Info section. Extend it. You should see your Nvidia GPU listed here.

• Scroll further to Docker Info and extend it. Verify that the nvidia runtime is recognized by Kasm.

Install the Kasm AI Registry (Optional, but Recommended for AI/ML workflows):

• Go to the Registries tab in the Kasm admin UI.
• If you are on Kasm version 1.17 or newer, the Kasm AI Registry should be visible in the “Registry Spotlight.” Click Install.

• This registry contains pre-configured workspaces with GPU support for various AI/ML tools.

Enable GPU for Workspaces:

By default, all images from the Kasm AI Registry have GPU support enabled. For other workspaces (e.g., a standard Ubuntu desktop or Blender from the default registry), you need to enable GPU access manually:

• Go to Workspaces.
• Find the workspace you want to configure, click its menu (three dots), and select Edit.
• In the workspace settings, set the GPU Count to 1. You can increase this if you want to pass through multiple GPUs to a single session (though this implies multiple physical GPUs passed to the VM or vGPU configurations).
• Save the changes.

Agent GPU Override (Advanced): You can configure the Kasm Agent to allow multiple container sessions to share the same physical GPU.

• Go to Infrastructure → Docker Agents.
• Edit your agent.
• Set the GPU Override setting. For example, setting it to 4 means up to four container sessions can attempt to use the same GPU.
• Note: This does not evenly distribute GPU resources. For fine-grained resource allocation, vGPU solutions are required.

Step 5: Launching and Verifying GPU-Accelerated Workspaces

Basic GPU Test in a Standard Workspace:

• Launch a workspace for which you’ve enabled GPU (e.g., a Kasm Ubuntu Desktop).
• Open a terminal within the Kasm session and run nvidia-smi. You should see the GPU details, confirming the GPU is accessible within the container.

• You can also launch Firefox within the workspace and run WebGL browser tests (e.g., webglsamples.org/aquarium/aquarium.html) to confirm GPU-accelerated rendering. (Note: Chrome/Chromium-based browsers in Kasm might not have full GPU acceleration support for rendering at the time of writing; this is being worked on.)

Showcasing Kasm AI Registry Workspaces:

Easy Diffusion:

• Install the “Easy Diffusion” workspace from the Kasm AI Registry.
• Launch it. This workspace comes pre-loaded with all necessary tools.
• The environment will set up, and Easy Diffusion’s web interface will automatically launch in a browser within the Kasm session. You can start generating images from text prompts, leveraging the GPU for fast processing.

CUDA-enabled PyTorch:

• Install and launch the “CUDA-enabled PyTorch” workspace from the Kasm AI Registry.
• This workspace includes PyTorch pre-installed.
• Open a terminal or Python environment and verify PyTorch can detect the GPU:

import torch
print(f"PyTorch version: {torch.__version__}")
print(f"CUDA available: {torch.cuda.is_available()}")
if torch.cuda.is_available():
    print(f"CUDA version: {torch.version.cuda}")
    print(f"GPU Name: {torch.cuda.get_device_name(0)}") print(f"CUDA available: {torch.cuda.is_available()}") if torch.cuda.is_available():     print(f"CUDA version: {torch.version.cuda}")     print(f"GPU Name: {torch.cuda.get_device_name(0)}")

• This output will confirm that PyTorch is successfully utilizing the GPU.

Blender with GPU Acceleration:

• The standard “Blender” workspace may not have GPU enabled by default.
• Edit its settings as described in Step 4.3 to set “GPU Count” to 1.
• Launch Blender.
• Inside Blender, navigate to Edit → Preferences → System.
• Under Cycles Render Devices, select CUDA or OptiX and ensure your Nvidia GPU is checked.
• You can now use Blender for your VFX workflows with GPU-accelerated rendering in Cycles.

GPU Workloads

By following these steps, you have successfully configured PCI passthrough for an Nvidia GPU on your Harvester hypervisor, installed the necessary drivers and runtime in your Kasm-hosting VM, and enabled GPU acceleration for Kasm Workspaces. Your Kasm deployment is now equipped to handle demanding GPU-accelerated workloads, providing a powerful and flexible container streaming solution.