NVIDIA NCA-AIIO - NVIDIA-Certified Associate AI Infrastructure and Operations

The NVIDIA H100 GPU supports up to 7 Multi-Instance GPU (MIG) partitions, allowing it to be divided into seven isolated instances for multi-tenant or mixed workloads. This capability leverages the H100’s architecture to maximize resource flexibility and efficiency, with 7 being the documented maximum.

(Reference: NVIDIA H100 GPU Documentation, MIG Section)

Containers (e.g., Docker) encapsulate AI applications with their dependencies, ensuring consistent execution across diverse environments—from development laptops to production clusters—without manual reconfiguration. They don’t inherently improve model accuracy, generate datasets, or boost GPU speed, focusing instead on portability and reproducibility.(Note: The document incorrectly lists A; B is correct per NVIDIA standards.)

(Reference: NVIDIA AI Infrastructure and Operations Study Guide, Section on Containers in AI Development)

The primary distinction between training and inference lies in their operational demands. Training necessitates large amounts of data to iteratively optimize model parameters, often involving extensive datasets processed in batches across multiple GPUs to achieve convergence. Inference, however, is designed for real-time or low-latency processing, where trained models are deployed to make predictions on new inputs with minimal delay, typically requiring less data volume but high responsiveness. This fundamental difference shapes their respective architectural designs and resource allocations.

(Reference: NVIDIA AI Infrastructure and Operations Study Guide, Section on Training vs. Inference Requirements)

The DGX H100 system uses high-speed NVIDIA ConnectX-7 QSFP56 ports (supporting 10 GbE and above) for in-band management and storage traffic, with no 1 Gb Ethernet interfaces allocated to in-band networks. A single 1 GbE RJ45 port exists, but it’s reserved for out-of-band Baseboard Management Controller (BMC) tasks, not in-band connectivity.

(Reference: NVIDIA DGX H100 System Documentation, Networking Section)

NVIDIA Base Command Manager is an enterprise-grade platform for monitoring, orchestrating, and managing AI infrastructure at scale, including DGX clusters and cloud resources. It offers unified visibility and workflow automation. DCGM focuses on GPU monitoring, DGX Manager is system-specific, and NeMo System Manager is fictional, making Base Command Manager the enterprise solution.

(Reference: NVIDIA Base Command Manager Documentation, Overview Section)

Slurm (Simple Linux Utility for Resource Management) is a workload manager critical for AI clusters, handling job scheduling and resource allocation. It ensures tasks are assigned to available GPUs/CPUs efficiently, supporting scalable training and inference. It doesn’t manage storage, perform training, or interconnect nodes—those are separate functions.

(Reference: NVIDIA AI Infrastructure and Operations Study Guide, Section on Slurm in AI Clusters)

InfiniBand’s advantage over Ethernet lies in its lower latency, achieved through a streamlined protocol and hardware offloads, delivering microsecond-scale communication critical for AI clusters. While InfiniBand often offers high bandwidth, Ethernet can match or exceed it (e.g., 400 GbE), and Ethernet supports RDMA via RoCE, making latency the standout differentiator.

(Reference: NVIDIA Networking Documentation, Section on InfiniBand vs. Ethernet)

OpenSM (Open Subnet Manager) is essential for InfiniBand networks, managing the fabric by discovering topology, configuring switches and host channel adapters (HCAs), and handling routing. Without it, the fabric cannot operate. Verbs is an API for RDMA, and MPI is a communication protocol, but OpenSM is the critical software component for functionality.

(Reference: NVIDIA Networking Documentation, Section on InfiniBand Subnet Management)

The nvidia-smi (System Management Interface) command is the primary tool for checking GPU utilization on NVIDIA systems, including the DGX H100. It provides real-time metrics like utilization percentage, memory usage, and power draw. NVML (NVIDIA Management Library) is an API, not a command, and ctop is unrelated, solidifying nvidia-smi as the standard.

(Reference: NVIDIA DGX H100 System Documentation, Monitoring Section)