NVIDIA DGX Rubin NVL8 Technical Analysis: AI Training and Inference Accelerator

As large-scale models gradually transition from the research phase to large-scale deployment, the focus of AI infrastructure is shifting. Over the past few years, the industry has primarily competed around training efficiency — larger model sizes, higher compute density, and faster training speeds. However, with the expansion of model scale and application scenarios, enterprises are increasingly shifting their focus toward inference performance, resource utilization efficiency, and real-world system performance in production environments. The design of NVIDIA DGX Rubin NVL8 reflects this trend. This article provides an analysis of DGX Rubin NVL8 from the perspectives of architectural design and performance characteristics.

The Shift in AI Infrastructure Focus: From Training to Inference

Traditional AI infrastructure development has mainly focused on training tasks, with optimization directions including increasing computational power, expanding memory capacity, and scaling network bandwidth. Key metrics during the training phase typically include training samples per second, gradient synchronization speed, and the stability of large-scale model training.

However, as large models become more widely adopted in enterprise applications, inference performance has become a critical factor in evaluating system value. Inference workloads typically involve the following characteristics:

• High-concurrency request processing
• Low-latency response
• Multi-model parallel execution

These characteristics impose different efficiency requirements on the system. Pure training capability alone is insufficient to support high-concurrency inference demands. Therefore, modern AI infrastructure must balance resource allocation and scheduling efficiency for both training and inference.

The design of DGX Rubin NVL8 fully embodies this trend. The system not only optimizes training throughput but also improves data flow, memory access paths, and task scheduling strategies, enabling it to maintain excellent performance under high-concurrency, multi-model inference scenarios. This “training–inference balance” design philosophy highlights the transition of AI infrastructure from compute-centric scaling to system-level efficiency optimization.

From Blackwell to Rubin: The Evolution of NVIDIA GPU Architectures

To understand DGX Rubin NVL8, it is necessary to review the evolution of NVIDIA GPU architectures. From Volta to Turing, followed by Ampere, Hopper, and Blackwell, each generation has continuously improved AI compute density, memory bandwidth, and multi-GPU interconnect capabilities while maintaining compatibility with the CUDA ecosystem.

So why is the new generation named Rubin rather than Blackwell Ultra? The answer lies in several key technological breakthroughs:

• Process advancement: Rubin adopts TSMC’s 3nm process. Compared with Blackwell’s 4nm process, it achieves higher transistor density and improved power efficiency.
• Memory architecture innovation: The introduction of HBM4 memory requires a redesigned memory controller and optimized data paths to meet higher bandwidth demands.
• Interconnect upgrade: NVLink 6.0 doubles bandwidth, with redesigned SerDes circuits and protocol stack.

These technological iterations collectively make Rubin a new architecture rather than a simple upgrade of Blackwell, providing a more efficient foundation for large-scale model training and inference.

DGX Rubin NVL8 Core Architecture Analysis

DGX Rubin NVL8 is a high-performance multi-GPU platform built on the NVIDIA Rubin GPU architecture, designed to support large-scale model training and inference workloads. Its system architecture includes GPUs, CPUs, interconnect technologies, storage, and network interfaces, all optimized at the system level to improve overall performance and resource utilization.

Rubin GPUs

Within the DGX Rubin NVL8 system, Rubin GPUs serve as the core compute units, optimized for both large-scale model training and inference. Each GPU is equipped with HBM3e memory and multi-level cache, supporting high-density matrix operations and sparse computing, which improves training efficiency while ensuring low-latency data access in inference scenarios.

The system integrates 8 Rubin GPUs, with a total memory capacity of up to 2.3 TB and memory bandwidth reaching 160 TB/s. Performance metrics are as follows:

• NVFP4 inference mode: 400 PFLOPS
• NVFP4 training mode: 280 PFLOPS
• FP8/FP6 training mode: 140 PFLOPS

Intel Xeon 6 Processors

The CPU subsystem consists of two Intel Xeon 6776P processors, supporting high-concurrency data processing and task scheduling. The multi-core, high-frequency configuration enables efficient handling of data preprocessing, task allocation, and GPU management.

Optimized memory channels and PCIe 6.0 interfaces ensure low-latency data transfer between CPU and GPU, providing stable support for both training and inference workloads.

Within DGX Rubin NVL8, CPUs collaborate with GPUs to manage tasks and schedule resources, ensuring efficient execution of training and inference workloads while also handling data loading and preprocessing.

6th Generation NVIDIA NVLink

DGX Rubin NVL8 adopts the 6th generation NVLink interconnect technology, providing up to 28.8 TB/s of bidirectional bandwidth across 8 GPUs. In multi-GPU collaborative inference scenarios, NVLink enables near lossless memory sharing between GPUs, supporting more efficient tensor parallelism strategies.

This ultra-high bandwidth, ultra-low latency interconnect architecture allows the 8 GPUs to function logically as a unified compute engine with a massive shared memory pool, enabling smooth execution of trillion-parameter models within a single node.

Storage and Network Interfaces

DGX Rubin NVL8 is equipped with 8 OSFP ports, each connected to a single-port NVIDIA ConnectX-9 VPI, delivering a total bandwidth of up to 800 Gb/s. In addition, the system includes two 400G QSP112 NVIDIA BlueField-4 DPUs, supporting both InfiniBand and Ethernet protocols, making it suitable for multi-node distributed training and inference deployments.

Conclusion

The DGX Rubin NVL8 reflects a technological development direction for AI infrastructure: achieving a balance between training and inference tasks, and improving overall resource utilization efficiency through system-level optimization. The high computational density of the Rubin GPU, the high-speed interconnect of NVLink 6.0, and the collaborative design of CPU, storage, and network provide a scalable and efficient hardware platform for large model training and inference.