NVIDIA NCP-AII Practice Test 119
Question 1180
During an NVIDIA AI factory acceptance test, engineers discover: NCCL training hangs only at peak concurrency while InfiniBand counters reveal shared-switch congestion events. Which TWO complementary elements are needed: the correct technical action and the strongest independent diagnostic evidence? Select TWO answers.
- Review kernel and NVIDIA Xid logs, driver version, concurrent workload, ECC state and vendor diagnostic guidance.
- Compare NCCL logs, per-hop switch telemetry, PFC or ECN indicators, port errors and concurrent job timing.
- Trace communication bottlenecks through topology, counters, congestion signals and workload concurrency before targeted tuning.
- Drain workloads and follow the vendor's approved component replacement, firmware and postrepair diagnostic runbook.
- Diagnose every NCCL hang as a faulty GPU and ignore correlated switch queue or fabric pause events.
Explanation
Correct Answers
B:This is the direct operational evidence to investigate: Compare NCCL logs, per-hop switch telemetry, PFC or ECN indicators, port errors and concurrent job timing. Observed scenario: NCCL training hangs only at peak concurrency while InfiniBand counters reveal shared-switch congestion events. The governing mechanism is: Interconnect saturation and pause behavior can cause collective stalls even when all hosts and links appear Up.
C:This is an appropriate correction for Network fabric congestion and intermittent collective hangs: Trace communication bottlenecks through topology, counters, congestion signals and workload concurrency before targeted tuning. Observed scenario: NCCL training hangs only at peak concurrency while InfiniBand counters reveal shared-switch congestion events. The governing mechanism is: Interconnect saturation and pause behavior can cause collective stalls even when all hosts and links appear Up.
Incorrect Answers
A:This alternative is relevant to NVIDIA GPU Xid event root cause isolation but does not directly resolve the current Network fabric congestion and intermittent collective hangs issue: Review kernel and NVIDIA Xid logs, driver version, concurrent workload, ECC state and vendor diagnostic guidance. Observed scenario: NCCL training hangs only at peak concurrency while InfiniBand counters reveal shared-switch congestion events. The governing mechanism is: Interconnect saturation and pause behavior can cause collective stalls even when all hosts and links appear Up.
D:This alternative is relevant to GPU and NVSwitch field replaceable component workflow but does not directly resolve the current Network fabric congestion and intermittent collective hangs issue: Drain workloads and follow the vendor's approved component replacement, firmware and postrepair diagnostic runbook. Observed scenario: NCCL training hangs only at peak concurrency while InfiniBand counters reveal shared-switch congestion events.
E:This tempting but unsafe shortcut is not a sufficient remediation for Network fabric congestion and intermittent collective hangs: Diagnose every NCCL hang as a faulty GPU and ignore correlated switch queue or fabric pause events. NCCL training hangs only at peak concurrency while InfiniBand counters reveal shared-switch congestion events. Interconnect saturation and pause behavior can cause collective stalls even when all hosts and links appear Up.
Question 1181
A production GPU cluster review identifies this condition: A RoCE fabric shows high average throughput but recurrent pauses that cause severe all-reduce tail latency. Which operational response is MOST appropriate to address the root issue? Select ONE best answer.
- Diagnose every NCCL hang as a faulty GPU and ignore correlated switch queue or fabric pause events.
- Trace communication bottlenecks through topology, counters, congestion signals and workload concurrency before targeted tuning.
- Measure locality and CPU-memory traffic, then use approved AMD platform NUMA and affinity settings for the workload.
- Correlate cooling airflow, fan health, ambient inlet and clock-throttle reasons; repair root cause safely.
Explanation
Correct Answer
B:This is an appropriate correction for Network fabric congestion and intermittent collective hangs: Trace communication bottlenecks through topology, counters, congestion signals and workload concurrency before targeted tuning. Observed scenario: A RoCE fabric shows high average throughput but recurrent pauses that cause severe all-reduce tail latency.
Incorrect Answers
A:This tempting but unsafe shortcut is not a sufficient remediation for Network fabric congestion and intermittent collective hangs: Diagnose every NCCL hang as a faulty GPU and ignore correlated switch queue or fabric pause events. A RoCE fabric shows high average throughput but recurrent pauses that cause severe all-reduce tail latency.
C:This alternative is relevant to AMD host CPU NUMA and memory tuning but does not directly resolve the current Network fabric congestion and intermittent collective hangs issue: Measure locality and CPU-memory traffic, then use approved AMD platform NUMA and affinity settings for the workload. Observed scenario: A RoCE fabric shows high average throughput but recurrent pauses that cause severe all-reduce tail latency.
D:This alternative is relevant to Fan faults and thermal throttling investigation but does not directly resolve the current Network fabric congestion and intermittent collective hangs issue: Correlate cooling airflow, fan health, ambient inlet and clock-throttle reasons; repair root cause safely. Observed scenario: A RoCE fabric shows high average throughput but recurrent pauses that cause severe all-reduce tail latency.
Question 1182
In a professional AI infrastructure deployment, a technician observes: A RoCE fabric shows high average throughput but recurrent pauses that cause severe all-reduce tail latency. Which underlying infrastructure principle BEST explains the issue? Select ONE best answer.
- Storage resilience introduces capacity and performance tradeoffs that matter during degradation and recovery.
- Interconnect saturation and pause behavior can cause collective stalls even when all hosts and links appear Up.
- Server CPU socket topology, PCIe lane allocation and memory access shape the efficiency of host-to-GPU transfers.
- Diagnose every NCCL hang as a faulty GPU and ignore correlated switch queue or fabric pause events.
Explanation
Correct Answer
B:This is the governing infrastructure principle: Interconnect saturation and pause behavior can cause collective stalls even when all hosts and links appear Up. Observed scenario: A RoCE fabric shows high average throughput but recurrent pauses that cause severe all-reduce tail latency.
Incorrect Answers
A:This alternative is relevant to Storage array throughput and RAID or erasure recovery but does not directly resolve the current Network fabric congestion and intermittent collective hangs issue: Storage resilience introduces capacity and performance tradeoffs that matter during degradation and recovery. Observed scenario: A RoCE fabric shows high average throughput but recurrent pauses that cause severe all-reduce tail latency.
C:That option concerns Intel host CPU PCIe placement and BIOS efficiency. The acceptance condition is: Sustained multi-node collectives meet approved latency and throughput targets without recurrent congestive stalls.
D:This tempting but unsafe shortcut is not a sufficient remediation for Network fabric congestion and intermittent collective hangs: Diagnose every NCCL hang as a faulty GPU and ignore correlated switch queue or fabric pause events. A RoCE fabric shows high average throughput but recurrent pauses that cause severe all-reduce tail latency. Interconnect saturation and pause behavior can cause collective stalls even when all hosts and links appear Up.
Question 1183
The cluster operations team needs to evaluate this situation: A RoCE fabric shows high average throughput but recurrent pauses that cause severe all-reduce tail latency. Which diagnostic evidence should be obtained FIRST to substantiate the likely fault or configuration gap? Select ONE best answer.
- Diagnose every NCCL hang as a faulty GPU and ignore correlated switch queue or fabric pause events.
- Inspect asset identifiers, FRU service records, firmware compatibility, PCIe enumeration and NVLink or NCCL results.
- Inspect lscpu NUMA topology, numactl placement, memory bandwidth, CPU affinity and GPU data-feed stalls.
- Compare NCCL logs, per-hop switch telemetry, PFC or ECN indicators, port errors and concurrent job timing.
Explanation
Correct Answer
D:This is the direct operational evidence to investigate: Compare NCCL logs, per-hop switch telemetry, PFC or ECN indicators, port errors and concurrent job timing. Observed scenario: A RoCE fabric shows high average throughput but recurrent pauses that cause severe all-reduce tail latency.
Incorrect Answers
A:This tempting but unsafe shortcut is not a sufficient remediation for Network fabric congestion and intermittent collective hangs: Diagnose every NCCL hang as a faulty GPU and ignore correlated switch queue or fabric pause events. Observed scenario: A RoCE fabric shows high average throughput but recurrent pauses that cause severe all-reduce tail latency. The governing mechanism is: Interconnect saturation and pause behavior can cause collective stalls even when all hosts and links appear Up. The proportionate action is: Trace communication bottlenecks through topology, counters, congestion signals and workload concurrency before targeted tuning.
B:This alternative is relevant to GPU and NVSwitch field replaceable component workflow but does not directly resolve the current Network fabric congestion and intermittent collective hangs issue: Inspect asset identifiers, FRU service records, firmware compatibility, PCIe enumeration and NVLink or NCCL results. Observed scenario: A RoCE fabric shows high average throughput but recurrent pauses that cause severe all-reduce tail latency.
C:This alternative is relevant to AMD host CPU NUMA and memory tuning but does not directly resolve the current Network fabric congestion and intermittent collective hangs issue: Inspect lscpu NUMA topology, numactl placement, memory bandwidth, CPU affinity and GPU data-feed stalls. Observed scenario: A RoCE fabric shows high average throughput but recurrent pauses that cause severe all-reduce tail latency.
Question 1184
At a multi-node AI infrastructure handoff, the team records: A RoCE fabric shows high average throughput but recurrent pauses that cause severe all-reduce tail latency. After corrective work, which observable result BEST confirms that the requirement is met? Select ONE best answer.
- The identified cause is addressed and repeated representative work runs without the original Xid failure.
- The new component is correctly inventoried and passes supported compute and interconnect tests before rescheduling.
- Diagnose every NCCL hang as a faulty GPU and ignore correlated switch queue or fabric pause events.
- Sustained multi-node collectives meet approved latency and throughput targets without recurrent congestive stalls.
Explanation
Correct Answer
D:This is an observable verified result, not merely a claimed configuration change: Sustained multi-node collectives meet approved latency and throughput targets without recurrent congestive stalls. Observed scenario: A RoCE fabric shows high average throughput but recurrent pauses that cause severe all-reduce tail latency.
Incorrect Answers
A:This alternative is relevant to NVIDIA GPU Xid event root cause isolation but does not directly resolve the current Network fabric congestion and intermittent collective hangs issue: The identified cause is addressed and repeated representative work runs without the original Xid failure. Observed scenario: A RoCE fabric shows high average throughput but recurrent pauses that cause severe all-reduce tail latency.
B:This alternative is relevant to GPU and NVSwitch field replaceable component workflow but does not directly resolve the current Network fabric congestion and intermittent collective hangs issue: The new component is correctly inventoried and passes supported compute and interconnect tests before rescheduling. Observed scenario: A RoCE fabric shows high average throughput but recurrent pauses that cause severe all-reduce tail latency.
C:This tempting but unsafe shortcut is not a sufficient remediation for Network fabric congestion and intermittent collective hangs: Diagnose every NCCL hang as a faulty GPU and ignore correlated switch queue or fabric pause events. Observed scenario: A RoCE fabric shows high average throughput but recurrent pauses that cause severe all-reduce tail latency. The governing mechanism is: Interconnect saturation and pause behavior can cause collective stalls even when all hosts and links appear Up. The proportionate action is: Trace communication bottlenecks through topology, counters, congestion signals and workload concurrency before targeted tuning. A direct check is: Compare NCCL logs, per-hop switch telemetry, PFC or ECN indicators, port errors and concurrent job timing. The acceptance condition is: Sustained multi-node collectives meet approved latency and throughput targets without recurrent congestive stalls.
Question 1185
During an NVIDIA AI factory acceptance test, engineers discover: A network team reimages all workers before comparing the fabric error and congestion telemetry associated with slow jobs. Which operational response is MOST appropriate to address the root issue? Select ONE best answer.
- Localize HCA, port, optic, cabling and PCIe causes before replacing the approved failed component.
- Trace communication bottlenecks through topology, counters, congestion signals and workload concurrency before targeted tuning.
- Diagnose every NCCL hang as a faulty GPU and ignore correlated switch queue or fabric pause events.
- Correlate cooling airflow, fan health, ambient inlet and clock-throttle reasons; repair root cause safely.
Explanation
Correct Answer
B:This is an appropriate correction for Network fabric congestion and intermittent collective hangs: Trace communication bottlenecks through topology, counters, congestion signals and workload concurrency before targeted tuning. Observed scenario: A network team reimages all workers before comparing the fabric error and congestion telemetry associated with slow jobs.
Incorrect Answers
A:That option concerns Failed ConnectX HCA or port fault isolation. Interconnect saturation and pause behavior can cause collective stalls even when all hosts and links appear Up.
C:This tempting but unsafe shortcut is not a sufficient remediation for Network fabric congestion and intermittent collective hangs: Diagnose every NCCL hang as a faulty GPU and ignore correlated switch queue or fabric pause events. A network team reimages all workers before comparing the fabric error and congestion telemetry associated with slow jobs.
D:This alternative is relevant to Fan faults and thermal throttling investigation but does not directly resolve the current Network fabric congestion and intermittent collective hangs issue: Correlate cooling airflow, fan health, ambient inlet and clock-throttle reasons; repair root cause safely. Observed scenario: A network team reimages all workers before comparing the fabric error and congestion telemetry associated with slow jobs.
Question 1186
A production GPU cluster review identifies this condition: A network team reimages all workers before comparing the fabric error and congestion telemetry associated with slow jobs. Which underlying infrastructure principle BEST explains the issue? Select ONE best answer.
- Power-supply redundancy protects uptime only when remaining units and external feeds support actual operating demand.
- Interconnect saturation and pause behavior can cause collective stalls even when all hosts and links appear Up.
- Diagnose every NCCL hang as a faulty GPU and ignore correlated switch queue or fabric pause events.
- NVIDIA Xid messages signal GPU-related errors with varying causes and severity; the exact code and context matter.
Explanation
Correct Answer
B:This is the governing infrastructure principle: Interconnect saturation and pause behavior can cause collective stalls even when all hosts and links appear Up. Observed scenario: A network team reimages all workers before comparing the fabric error and congestion telemetry associated with slow jobs.
Incorrect Answers
A:That option concerns Redundant PSU and power path failure replacement. The acceptance condition is: Sustained multi-node collectives meet approved latency and throughput targets without recurrent congestive stalls.
C:This tempting but unsafe shortcut is not a sufficient remediation for Network fabric congestion and intermittent collective hangs: Diagnose every NCCL hang as a faulty GPU and ignore correlated switch queue or fabric pause events. A network team reimages all workers before comparing the fabric error and congestion telemetry associated with slow jobs. Interconnect saturation and pause behavior can cause collective stalls even when all hosts and links appear Up.
D:This alternative is relevant to NVIDIA GPU Xid event root cause isolation but does not directly resolve the current Network fabric congestion and intermittent collective hangs issue: NVIDIA Xid messages signal GPU-related errors with varying causes and severity; the exact code and context matter. Observed scenario: A network team reimages all workers before comparing the fabric error and congestion telemetry associated with slow jobs.
Question 1187
In a professional AI infrastructure deployment, a technician observes: A network team reimages all workers before comparing the fabric error and congestion telemetry associated with slow jobs. Which diagnostic evidence should be obtained FIRST to substantiate the likely fault or configuration gap? Select ONE best answer.
- Inspect nvidia-smi ECC history, DCGM alerts, remapped rows, Xid logs and stress test results.
- Inspect PSU health LEDs or BMC alerts, output power, PDU branches, hot-swap procedure and failover event logs.
- Diagnose every NCCL hang as a faulty GPU and ignore correlated switch queue or fabric pause events.
- Compare NCCL logs, per-hop switch telemetry, PFC or ECN indicators, port errors and concurrent job timing.
Explanation
Correct Answer
D:This is the direct operational evidence to investigate: Compare NCCL logs, per-hop switch telemetry, PFC or ECN indicators, port errors and concurrent job timing. Observed scenario: A network team reimages all workers before comparing the fabric error and congestion telemetry associated with slow jobs.
Incorrect Answers
A:This alternative is relevant to GPU ECC and memory reliability diagnosis but does not directly resolve the current Network fabric congestion and intermittent collective hangs issue: Inspect nvidia-smi ECC history, DCGM alerts, remapped rows, Xid logs and stress test results. Observed scenario: A network team reimages all workers before comparing the fabric error and congestion telemetry associated with slow jobs.
B:This alternative is relevant to Redundant PSU and power path failure replacement but does not directly resolve the current Network fabric congestion and intermittent collective hangs issue: Inspect PSU health LEDs or BMC alerts, output power, PDU branches, hot-swap procedure and failover event logs. Observed scenario: A network team reimages all workers before comparing the fabric error and congestion telemetry associated with slow jobs.
C:This tempting but unsafe shortcut is not a sufficient remediation for Network fabric congestion and intermittent collective hangs: Diagnose every NCCL hang as a faulty GPU and ignore correlated switch queue or fabric pause events. Observed scenario: A network team reimages all workers before comparing the fabric error and congestion telemetry associated with slow jobs. The governing mechanism is: Interconnect saturation and pause behavior can cause collective stalls even when all hosts and links appear Up. The proportionate action is: Trace communication bottlenecks through topology, counters, congestion signals and workload concurrency before targeted tuning.
Question 1188
The cluster operations team needs to evaluate this situation: A network team reimages all workers before comparing the fabric error and congestion telemetry associated with slow jobs. After corrective work, which observable result BEST confirms that the requirement is met? Select ONE best answer.
- Intended GPU and NIC paths deliver stable accepted bandwidth and CPU overhead under sustained representative load.
- Sustained multi-node collectives meet approved latency and throughput targets without recurrent congestive stalls.
- Diagnose every NCCL hang as a faulty GPU and ignore correlated switch queue or fabric pause events.
- The device either passes approved health criteria under load or is replaced with traceable fault closure.
Explanation
Correct Answer
B:This is an observable verified result, not merely a claimed configuration change: Sustained multi-node collectives meet approved latency and throughput targets without recurrent congestive stalls. Observed scenario: A network team reimages all workers before comparing the fabric error and congestion telemetry associated with slow jobs.
Incorrect Answers
A:That option concerns Intel host CPU PCIe placement and BIOS efficiency. Interconnect saturation and pause behavior can cause collective stalls even when all hosts and links appear Up.
C:This tempting but unsafe shortcut is not a sufficient remediation for Network fabric congestion and intermittent collective hangs: Diagnose every NCCL hang as a faulty GPU and ignore correlated switch queue or fabric pause events. Observed scenario: A network team reimages all workers before comparing the fabric error and congestion telemetry associated with slow jobs. The governing mechanism is: Interconnect saturation and pause behavior can cause collective stalls even when all hosts and links appear Up. The proportionate action is: Trace communication bottlenecks through topology, counters, congestion signals and workload concurrency before targeted tuning. A direct check is: Compare NCCL logs, per-hop switch telemetry, PFC or ECN indicators, port errors and concurrent job timing. The acceptance condition is: Sustained multi-node collectives meet approved latency and throughput targets without recurrent congestive stalls.
D:This alternative is relevant to GPU ECC and memory reliability diagnosis but does not directly resolve the current Network fabric congestion and intermittent collective hangs issue: The device either passes approved health criteria under load or is replaced with traceable fault closure. Observed scenario: A network team reimages all workers before comparing the fabric error and congestion telemetry associated with slow jobs.
Question 1189
At a multi-node AI infrastructure handoff, the team records: A cluster stops serving jobs after several simultaneous firmware, driver and network changes with no recorded baseline. Which operational response is MOST appropriate to address the root issue? Select ONE best answer.
- Isolate the failed power path, verify redundancy budget and follow supported PSU replacement with monitoring.
- Quarantine and diagnose affected GPUs using ECC, row-remap and diagnostic evidence before authorized repair or return.
- Make broad unrelated configuration changes during a cluster outage without capturing logs, state or a rollback point.
- Use a staged evidence-led incident workflow, isolate the nearest failing layer and validate a safe known-good rollback.
Explanation
Correct Answer
D:This is an appropriate correction for Operational fault isolation and safe rollback: Use a staged evidence-led incident workflow, isolate the nearest failing layer and validate a safe known-good rollback. Observed scenario: A cluster stops serving jobs after several simultaneous firmware, driver and network changes with no recorded baseline.
Incorrect Answers
A:This alternative is relevant to Redundant PSU and power path failure replacement but does not directly resolve the current Operational fault isolation and safe rollback issue: Isolate the failed power path, verify redundancy budget and follow supported PSU replacement with monitoring. Observed scenario: A cluster stops serving jobs after several simultaneous firmware, driver and network changes with no recorded baseline.
B:This alternative is relevant to GPU ECC and memory reliability diagnosis but does not directly resolve the current Operational fault isolation and safe rollback issue: Quarantine and diagnose affected GPUs using ECC, row-remap and diagnostic evidence before authorized repair or return. Observed scenario: A cluster stops serving jobs after several simultaneous firmware, driver and network changes with no recorded baseline.
C:This tempting but unsafe shortcut is not a sufficient remediation for Operational fault isolation and safe rollback: Make broad unrelated configuration changes during a cluster outage without capturing logs, state or a rollback point.
Question 1190
During an NVIDIA AI factory acceptance test, engineers discover: A cluster stops serving jobs after several simultaneous firmware, driver and network changes with no recorded baseline. Which TWO complementary elements are needed: the correct technical action and the strongest independent diagnostic evidence? Select TWO answers.
- Measure locality and CPU-memory traffic, then use approved AMD platform NUMA and affinity settings for the workload.
- Make broad unrelated configuration changes during a cluster outage without capturing logs, state or a rollback point.
- Review kernel and NVIDIA Xid logs, driver version, concurrent workload, ECC state and vendor diagnostic guidance.
- Review incident timeline, change diffs, hardware alarms, cluster-manager and workload logs, plus rollback comparison.
- Use a staged evidence-led incident workflow, isolate the nearest failing layer and validate a safe known-good rollback.
Explanation
Correct Answers
D:This is the direct operational evidence to investigate: Review incident timeline, change diffs, hardware alarms, cluster-manager and workload logs, plus rollback comparison. Observed scenario: A cluster stops serving jobs after several simultaneous firmware, driver and network changes with no recorded baseline.
E:This is an appropriate correction for Operational fault isolation and safe rollback: Use a staged evidence-led incident workflow, isolate the nearest failing layer and validate a safe known-good rollback. Observed scenario: A cluster stops serving jobs after several simultaneous firmware, driver and network changes with no recorded baseline. The governing mechanism is: Complex AI infrastructure failures span facility, nodes, fabrics, storage and control plane; uncontrolled changes hide cause.
Incorrect Answers
A:This alternative is relevant to AMD host CPU NUMA and memory tuning but does not directly resolve the current Operational fault isolation and safe rollback issue: Measure locality and CPU-memory traffic, then use approved AMD platform NUMA and affinity settings for the workload. Observed scenario: A cluster stops serving jobs after several simultaneous firmware, driver and network changes with no recorded baseline.
B:This tempting but unsafe shortcut is not a sufficient remediation for Operational fault isolation and safe rollback: Make broad unrelated configuration changes during a cluster outage without capturing logs, state or a rollback point. Complex AI infrastructure failures span facility, nodes, fabrics, storage and control plane; uncontrolled changes hide cause.
C:That option concerns NVIDIA GPU Xid event root cause isolation. Complex AI infrastructure failures span facility, nodes, fabrics, storage and control plane; uncontrolled changes hide cause.