NVIDIA NCP-AII Practice Test 99
Question 980
During an NVIDIA AI factory acceptance test, engineers discover: NCCL hangs periodically at high concurrency even though two-node quick tests pass. Which TWO complementary elements are needed: the correct technical action and the strongest independent diagnostic evidence? Select TWO answers.
- Run repeatable multi-node NCCL stress over representative durations, message sizes and full-rack participation.
- Inspect dcgmi diag suite results, entity IDs, error logs, driver state and conflicting running workloads.
- Validate collective correctness over representative sizes and iteration counts alongside throughput.
- Inspect long-run NCCL logs, error rate, interconnect counters, rank liveness and timing variation.
- Treat a one-second two-node NCCL demonstration as proof that a large distributed training fabric is burn-in qualified.
Explanation
Correct Answers
A:This is an appropriate correction for Multi-node NCCL repeated burn-in: Run repeatable multi-node NCCL stress over representative durations, message sizes and full-rack participation. Observed scenario: NCCL hangs periodically at high concurrency even though two-node quick tests pass. The governing mechanism is: Repeated collective communication can uncover unstable links, transport state and intermittent rank failures that smoke tests miss.
D:This is the direct operational evidence to investigate: Inspect long-run NCCL logs, error rate, interconnect counters, rank liveness and timing variation. Observed scenario: NCCL hangs periodically at high concurrency even though two-node quick tests pass. The governing mechanism is: Repeated collective communication can uncover unstable links, transport state and intermittent rank failures that smoke tests miss.
Incorrect Answers
B:This alternative is relevant to DCGM active single-node diagnostic suites but does not directly resolve the current Multi-node NCCL repeated burn-in issue: Inspect dcgmi diag suite results, entity IDs, error logs, driver state and conflicting running workloads. Observed scenario: NCCL hangs periodically at high concurrency even though two-node quick tests pass.
C:This alternative is relevant to NCCL device buffers and data integrity checks but does not directly resolve the current Multi-node NCCL repeated burn-in issue: Validate collective correctness over representative sizes and iteration counts alongside throughput. Observed scenario: NCCL hangs periodically at high concurrency even though two-node quick tests pass.
E:This tempting but unsafe shortcut is not a sufficient remediation for Multi-node NCCL repeated burn-in: Treat a one-second two-node NCCL demonstration as proof that a large distributed training fabric is burn-in qualified. Observed scenario: NCCL hangs periodically at high concurrency even though two-node quick tests pass. The governing mechanism is: Repeated collective communication can uncover unstable links, transport state and intermittent rank failures that smoke tests miss. The proportionate action is: Run repeatable multi-node NCCL stress over representative durations, message sizes and full-rack participation. A direct check is: Inspect long-run NCCL logs, error rate, interconnect counters, rank liveness and timing variation. The acceptance condition is: Every planned rank completes sustained correct collectives with throughput and error levels within acceptance limits.
Question 981
A production GPU cluster review identifies this condition: A topology upgrade changes network flow patterns but the old short benchmark is reused as proof of readiness. Which operational response is MOST appropriate to address the root issue? Select ONE best answer.
- Run repeatable multi-node NCCL stress over representative durations, message sizes and full-rack participation.
- Run a controlled representative GPU stress with monitoring of temperatures, memory and power-limit behavior.
- Treat a one-second two-node NCCL demonstration as proof that a large distributed training fabric is burn-in qualified.
- Run supported ClusterKit or equivalent multifaceted qualification across the intended node set and preserve evidence.
Explanation
Correct Answer
A:This is an appropriate correction for Multi-node NCCL repeated burn-in: Run repeatable multi-node NCCL stress over representative durations, message sizes and full-rack participation. Observed scenario: A topology upgrade changes network flow patterns but the old short benchmark is reused as proof of readiness.
Incorrect Answers
B:That option concerns GPU thermal power and memory single-node burn-in. Repeated collective communication can uncover unstable links, transport state and intermittent rank failures that smoke tests miss.
C:This tempting but unsafe shortcut is not a sufficient remediation for Multi-node NCCL repeated burn-in: Treat a one-second two-node NCCL demonstration as proof that a large distributed training fabric is burn-in qualified. A topology upgrade changes network flow patterns but the old short benchmark is reused as proof of readiness.
D:This alternative is relevant to ClusterKit multi-node qualification workflow but does not directly resolve the current Multi-node NCCL repeated burn-in issue: Run supported ClusterKit or equivalent multifaceted qualification across the intended node set and preserve evidence. Observed scenario: A topology upgrade changes network flow patterns but the old short benchmark is reused as proof of readiness.
Question 982
In a professional AI infrastructure deployment, a technician observes: A topology upgrade changes network flow patterns but the old short benchmark is reused as proof of readiness. Which underlying infrastructure principle BEST explains the issue? Select ONE best answer.
- Distributed HPL scaling depends on mapping processes and communications to actual CPU and GPU locality.
- Treat a one-second two-node NCCL demonstration as proof that a large distributed training fabric is burn-in qualified.
- Sustained accelerator load can reveal thermal and electrical limits not visible during idle or brief smoke checks.
- Repeated collective communication can uncover unstable links, transport state and intermittent rank failures that smoke tests miss.
Explanation
Correct Answer
D:This is the governing infrastructure principle: Repeated collective communication can uncover unstable links, transport state and intermittent rank failures that smoke tests miss. Observed scenario: A topology upgrade changes network flow patterns but the old short benchmark is reused as proof of readiness.
Incorrect Answers
A:That option concerns Multi-process HPL rank placement and CPU affinity. The acceptance condition is: Every planned rank completes sustained correct collectives with throughput and error levels within acceptance limits.
B:This tempting but unsafe shortcut is not a sufficient remediation for Multi-node NCCL repeated burn-in: Treat a one-second two-node NCCL demonstration as proof that a large distributed training fabric is burn-in qualified. A topology upgrade changes network flow patterns but the old short benchmark is reused as proof of readiness. Repeated collective communication can uncover unstable links, transport state and intermittent rank failures that smoke tests miss.
C:This alternative is relevant to GPU thermal power and memory single-node burn-in but does not directly resolve the current Multi-node NCCL repeated burn-in issue: Sustained accelerator load can reveal thermal and electrical limits not visible during idle or brief smoke checks. Observed scenario: A topology upgrade changes network flow patterns but the old short benchmark is reused as proof of readiness.
Question 983
The cluster operations team needs to evaluate this situation: A topology upgrade changes network flow patterns but the old short benchmark is reused as proof of readiness. Which diagnostic evidence should be obtained FIRST to substantiate the likely fault or configuration gap? Select ONE best answer.
- Inspect dcgmi diag suite results, entity IDs, error logs, driver state and conflicting running workloads.
- Treat a one-second two-node NCCL demonstration as proof that a large distributed training fabric is burn-in qualified.
- Inspect long-run NCCL logs, error rate, interconnect counters, rank liveness and timing variation.
- Inspect nccl-tests output verification flags, message sizes, GPU logs and reproducible error traces.
Explanation
Correct Answer
C:This is the direct operational evidence to investigate: Inspect long-run NCCL logs, error rate, interconnect counters, rank liveness and timing variation. Observed scenario: A topology upgrade changes network flow patterns but the old short benchmark is reused as proof of readiness.
Incorrect Answers
A:This alternative is relevant to DCGM active single-node diagnostic suites but does not directly resolve the current Multi-node NCCL repeated burn-in issue: Inspect dcgmi diag suite results, entity IDs, error logs, driver state and conflicting running workloads. Observed scenario: A topology upgrade changes network flow patterns but the old short benchmark is reused as proof of readiness.
B:This tempting but unsafe shortcut is not a sufficient remediation for Multi-node NCCL repeated burn-in: Treat a one-second two-node NCCL demonstration as proof that a large distributed training fabric is burn-in qualified. Observed scenario: A topology upgrade changes network flow patterns but the old short benchmark is reused as proof of readiness. The governing mechanism is: Repeated collective communication can uncover unstable links, transport state and intermittent rank failures that smoke tests miss. The proportionate action is: Run repeatable multi-node NCCL stress over representative durations, message sizes and full-rack participation.
D:This alternative is relevant to NCCL device buffers and data integrity checks but does not directly resolve the current Multi-node NCCL repeated burn-in issue: Inspect nccl-tests output verification flags, message sizes, GPU logs and reproducible error traces. Observed scenario: A topology upgrade changes network flow patterns but the old short benchmark is reused as proof of readiness.
Question 984
At a multi-node AI infrastructure handoff, the team records: A topology upgrade changes network flow patterns but the old short benchmark is reused as proof of readiness. After corrective work, which observable result BEST confirms that the requirement is met? Select ONE best answer.
- Treat a one-second two-node NCCL demonstration as proof that a large distributed training fabric is burn-in qualified.
- The specified node group completes its approved multifaceted readiness checks with traceable results.
- Every planned rank completes sustained correct collectives with throughput and error levels within acceptance limits.
- All targeted healthy GPUs complete the approved test suite or receive a documented fault disposition.
Explanation
Correct Answer
C:This is an observable verified result, not merely a claimed configuration change: Every planned rank completes sustained correct collectives with throughput and error levels within acceptance limits. Observed scenario: A topology upgrade changes network flow patterns but the old short benchmark is reused as proof of readiness.
Incorrect Answers
A:This tempting but unsafe shortcut is not a sufficient remediation for Multi-node NCCL repeated burn-in: Treat a one-second two-node NCCL demonstration as proof that a large distributed training fabric is burn-in qualified. Observed scenario: A topology upgrade changes network flow patterns but the old short benchmark is reused as proof of readiness. The governing mechanism is: Repeated collective communication can uncover unstable links, transport state and intermittent rank failures that smoke tests miss. The proportionate action is: Run repeatable multi-node NCCL stress over representative durations, message sizes and full-rack participation. A direct check is: Inspect long-run NCCL logs, error rate, interconnect counters, rank liveness and timing variation. The acceptance condition is: Every planned rank completes sustained correct collectives with throughput and error levels within acceptance limits.
B:This alternative is relevant to ClusterKit multi-node qualification workflow but does not directly resolve the current Multi-node NCCL repeated burn-in issue: The specified node group completes its approved multifaceted readiness checks with traceable results.
D:This alternative is relevant to DCGM active single-node diagnostic suites but does not directly resolve the current Multi-node NCCL repeated burn-in issue: All targeted healthy GPUs complete the approved test suite or receive a documented fault disposition. Observed scenario: A topology upgrade changes network flow patterns but the old short benchmark is reused as proof of readiness.
Question 985
During an NVIDIA AI factory acceptance test, engineers discover: A multi-hour HPL job eventually fails numerical residual checks although the first run passes. Which operational response is MOST appropriate to address the root issue? Select ONE best answer.
- Execute sustained HPL trials with validated numerical results, platform telemetry and consistent power constraints.
- Execute supported CUDA runtime initialization and minimal kernel tests on each scheduled accelerator.
- Report the best HPL number as full reliability evidence even if later long-run trials fail or throttle.
- Capture a reproducible preflight inventory of node hardware, software, connections and expected acceptance tests.
Explanation
Correct Answer
A:This is an appropriate correction for Sustained HPL burn-in thermal and numerical health: Execute sustained HPL trials with validated numerical results, platform telemetry and consistent power constraints. Observed scenario: A multi-hour HPL job eventually fails numerical residual checks although the first run passes.
Incorrect Answers
B:This alternative is relevant to CUDA device query and kernel smoke tests but does not directly resolve the current Sustained HPL burn-in thermal and numerical health issue: Execute supported CUDA runtime initialization and minimal kernel tests on each scheduled accelerator. Observed scenario: A multi-hour HPL job eventually fails numerical residual checks although the first run passes.
C:This tempting but unsafe shortcut is not a sufficient remediation for Sustained HPL burn-in thermal and numerical health: Report the best HPL number as full reliability evidence even if later long-run trials fail or throttle.
D:This alternative is relevant to Cluster-wide preflight hardware inventory but does not directly resolve the current Sustained HPL burn-in thermal and numerical health issue: Capture a reproducible preflight inventory of node hardware, software, connections and expected acceptance tests. Observed scenario: A multi-hour HPL job eventually fails numerical residual checks although the first run passes.
Question 986
A production GPU cluster review identifies this condition: A multi-hour HPL job eventually fails numerical residual checks although the first run passes. Which underlying infrastructure principle BEST explains the issue? Select ONE best answer.
- Report the best HPL number as full reliability evidence even if later long-run trials fail or throttle.
- High FLOPS does not prove durable cluster stability; repeated HPL can reveal thermal and computational errors.
- Some thermal, power, memory and fabric failures appear only under sustained stress; complete evidence includes trends.
- A representative performance benchmark is meaningful only when the tested hardware and software baseline are documented.
Explanation
Correct Answer
B:This is the governing infrastructure principle: High FLOPS does not prove durable cluster stability; repeated HPL can reveal thermal and computational errors. Observed scenario: A multi-hour HPL job eventually fails numerical residual checks although the first run passes.
Incorrect Answers
A:This tempting but unsafe shortcut is not a sufficient remediation for Sustained HPL burn-in thermal and numerical health: Report the best HPL number as full reliability evidence even if later long-run trials fail or throttle. High FLOPS does not prove durable cluster stability; repeated HPL can reveal thermal and computational errors.
C:This alternative is relevant to Long-duration burn-in schedule and symptom correlation but does not directly resolve the current Sustained HPL burn-in thermal and numerical health issue: Some thermal, power, memory and fabric failures appear only under sustained stress; complete evidence includes trends. Observed scenario: A multi-hour HPL job eventually fails numerical residual checks although the first run passes.
D:This alternative is relevant to Cluster-wide preflight hardware inventory but does not directly resolve the current Sustained HPL burn-in thermal and numerical health issue: A representative performance benchmark is meaningful only when the tested hardware and software baseline are documented. Observed scenario: A multi-hour HPL job eventually fails numerical residual checks although the first run passes.
Question 987
In a professional AI infrastructure deployment, a technician observes: A multi-hour HPL job eventually fails numerical residual checks although the first run passes. Which diagnostic evidence should be obtained FIRST to substantiate the likely fault or configuration gap? Select ONE best answer.
- Inspect node inventory, GPU and NIC UUIDs, firmware and driver versions, cabling records and test eligibility.
- Inspect per-iteration residual status, achieved FLOPS, GPU errors, thermal trend and job exit codes.
- Report the best HPL number as full reliability evidence even if later long-run trials fail or throttle.
- Review transceiver diagnostics, FEC corrected and uncorrectable counters, BER trends and optical power.
Explanation
Correct Answer
B:This is the direct operational evidence to investigate: Inspect per-iteration residual status, achieved FLOPS, GPU errors, thermal trend and job exit codes. Observed scenario: A multi-hour HPL job eventually fails numerical residual checks although the first run passes.
Incorrect Answers
A:This alternative is relevant to Cluster-wide preflight hardware inventory but does not directly resolve the current Sustained HPL burn-in thermal and numerical health issue: Inspect node inventory, GPU and NIC UUIDs, firmware and driver versions, cabling records and test eligibility. Observed scenario: A multi-hour HPL job eventually fails numerical residual checks although the first run passes.
C:This tempting but unsafe shortcut is not a sufficient remediation for Sustained HPL burn-in thermal and numerical health: Report the best HPL number as full reliability evidence even if later long-run trials fail or throttle. A multi-hour HPL job eventually fails numerical residual checks although the first run passes.
D:This alternative is relevant to Optical FEC BER validation during burn-in but does not directly resolve the current Sustained HPL burn-in thermal and numerical health issue: Review transceiver diagnostics, FEC corrected and uncorrectable counters, BER trends and optical power. Observed scenario: A multi-hour HPL job eventually fails numerical residual checks although the first run passes.
Question 988
The cluster operations team needs to evaluate this situation: A multi-hour HPL job eventually fails numerical residual checks although the first run passes. After corrective work, which observable result BEST confirms that the requirement is met? Select ONE best answer.
- The cluster completes the approved continuous HPL window with valid results and stable sustained performance.
- Each accepted HPL trial both passes numerical correctness and meets the approved performance threshold.
- Report the best HPL number as full reliability evidence even if later long-run trials fail or throttle.
- The target model workflow trains for the planned duration and produces recoverable checkpoints without systemic faults.
Explanation
Correct Answer
A:This is an observable verified result, not merely a claimed configuration change: The cluster completes the approved continuous HPL window with valid results and stable sustained performance. Observed scenario: A multi-hour HPL job eventually fails numerical residual checks although the first run passes.
Incorrect Answers
B:This alternative is relevant to HPL numerical result and residual validation but does not directly resolve the current Sustained HPL burn-in thermal and numerical health issue: Each accepted HPL trial both passes numerical correctness and meets the approved performance threshold. Observed scenario: A multi-hour HPL job eventually fails numerical residual checks although the first run passes.
C:This tempting but unsafe shortcut is not a sufficient remediation for Sustained HPL burn-in thermal and numerical health: Report the best HPL number as full reliability evidence even if later long-run trials fail or throttle. A multi-hour HPL job eventually fails numerical residual checks although the first run passes. High FLOPS does not prove durable cluster stability; repeated HPL can reveal thermal and computational errors.
D:This alternative is relevant to NeMo distributed training end-to-end burn-in but does not directly resolve the current Sustained HPL burn-in thermal and numerical health issue: The target model workflow trains for the planned duration and produces recoverable checkpoints without systemic faults. Observed scenario: A multi-hour HPL job eventually fails numerical residual checks although the first run passes.
Question 989
At a multi-node AI infrastructure handoff, the team records: A rack's GPU temperatures steadily rise during back-to-back HPL passes despite good average FLOPS. Which operational response is MOST appropriate to address the root issue? Select ONE best answer.
- Execute sustained HPL trials with validated numerical results, platform telemetry and consistent power constraints.
- Check the complete HPL result including residual validity, process exit status and repeatability before acceptance.
- Report the best HPL number as full reliability evidence even if later long-run trials fail or throttle.
- Plan approved-duration coordinated burn-in with continuous telemetry and clean test resource reservations.
Explanation
Correct Answer
A:This is an appropriate correction for Sustained HPL burn-in thermal and numerical health: Execute sustained HPL trials with validated numerical results, platform telemetry and consistent power constraints. Observed scenario: A rack's GPU temperatures steadily rise during back-to-back HPL passes despite good average FLOPS.
Incorrect Answers
B:This alternative is relevant to HPL numerical result and residual validation but does not directly resolve the current Sustained HPL burn-in thermal and numerical health issue: Check the complete HPL result including residual validity, process exit status and repeatability before acceptance. Observed scenario: A rack's GPU temperatures steadily rise during back-to-back HPL passes despite good average FLOPS.
C:This tempting but unsafe shortcut is not a sufficient remediation for Sustained HPL burn-in thermal and numerical health: Report the best HPL number as full reliability evidence even if later long-run trials fail or throttle. A rack's GPU temperatures steadily rise during back-to-back HPL passes despite good average FLOPS.
D:This alternative is relevant to Long-duration burn-in schedule and symptom correlation but does not directly resolve the current Sustained HPL burn-in thermal and numerical health issue: Plan approved-duration coordinated burn-in with continuous telemetry and clean test resource reservations. Observed scenario: A rack's GPU temperatures steadily rise during back-to-back HPL passes despite good average FLOPS.
Question 990
During an NVIDIA AI factory acceptance test, engineers discover: A rack's GPU temperatures steadily rise during back-to-back HPL passes despite good average FLOPS. Which TWO complementary elements are needed: the correct technical action and the strongest independent diagnostic evidence? Select TWO answers.
- Report the best HPL number as full reliability evidence even if later long-run trials fail or throttle.
- Inspect per-iteration residual status, achieved FLOPS, GPU errors, thermal trend and job exit codes.
- Inspect dcgmi diag suite results, entity IDs, error logs, driver state and conflicting running workloads.
- Collect optical power and FEC or BER data over representative load, then isolate unsafe optics or fibers.
- Execute sustained HPL trials with validated numerical results, platform telemetry and consistent power constraints.
Explanation
Correct Answers
B:This is the direct operational evidence to investigate: Inspect per-iteration residual status, achieved FLOPS, GPU errors, thermal trend and job exit codes. Observed scenario: A rack's GPU temperatures steadily rise during back-to-back HPL passes despite good average FLOPS.
E:This is an appropriate correction for Sustained HPL burn-in thermal and numerical health: Execute sustained HPL trials with validated numerical results, platform telemetry and consistent power constraints. Observed scenario: A rack's GPU temperatures steadily rise during back-to-back HPL passes despite good average FLOPS. The governing mechanism is: High FLOPS does not prove durable cluster stability; repeated HPL can reveal thermal and computational errors.
Incorrect Answers
A:This tempting but unsafe shortcut is not a sufficient remediation for Sustained HPL burn-in thermal and numerical health: Report the best HPL number as full reliability evidence even if later long-run trials fail or throttle. A rack's GPU temperatures steadily rise during back-to-back HPL passes despite good average FLOPS. High FLOPS does not prove durable cluster stability; repeated HPL can reveal thermal and computational errors.
C:That option concerns DCGM active single-node diagnostic suites. High FLOPS does not prove durable cluster stability; repeated HPL can reveal thermal and computational errors.
D:This alternative is relevant to Optical FEC BER validation during burn-in but does not directly resolve the current Sustained HPL burn-in thermal and numerical health issue: Collect optical power and FEC or BER data over representative load, then isolate unsafe optics or fibers. Observed scenario: A rack's GPU temperatures steadily rise during back-to-back HPL passes despite good average FLOPS.