NCCL

E209960

NCCL (NVIDIA Collective Communications Library) is a high-performance library that optimizes multi-GPU and multi-node communication for deep learning and HPC applications.

All labels observed (4)

Label Occurrences
NCCL canonical 6
NVIDIA NCCL 4
NVIDIA Collective Communications Library 1

How this entity was disambiguated

Statements (66)

Predicate Object
instanceOf GPU communication library ⓘ
collective communications library ⓘ
communication library ⓘ
parallel computing library ⓘ
developer NVIDIA ⓘ
linked to: NVIDIA Corporation
domain deep learning ⓘ
distributed training ⓘ
high-performance computing ⓘ
multi-GPU computing ⓘ
multi-node computing ⓘ
feature GPUDirect RDMA support ⓘ
asynchronous communication API ⓘ
grouped collective operations ⓘ
hierarchical collectives ⓘ
multi-process communication support ⓘ
ring-based collectives ⓘ
topology-aware communication algorithms ⓘ
tree-based collectives ⓘ
fullName NVIDIA Collective Communications Library ⓘ
linked to: NCCL
goal high-bandwidth GPU communication ⓘ
low-latency collective operations ⓘ
scalable multi-GPU training ⓘ
integratesWith CUDA ⓘ
linked to: NVIDIA CUDA
license BSD-style license ⓘ
optimizedFor GPU interconnects ⓘ
NVIDIA GPUs ⓘ
linked to: Nvidia Maxwell GPU
primaryAPIStyle C API ⓘ
programmingLanguage C ⓘ
provider NVIDIA ⓘ
linked to: NVIDIA Corporation
requires CUDA-enabled GPU ⓘ
supportsDataType bfloat16 ⓘ
float16 ⓘ
float32 ⓘ
float64 ⓘ
int32 ⓘ
int64 ⓘ
uint8 ⓘ
supportsEnvironment cloud GPU clusters ⓘ
on-premise clusters ⓘ
supportsHardware Ethernet ⓘ
InfiniBand ⓘ
NVLink ⓘ
linked to: NVLink interconnect

NVSwitch ⓘ
linked to: NVLink interconnect

PCIe ⓘ
linked to: PCI Express
supportsLanguageBinding C++ ⓘ
Python ⓘ
supportsOperation all-gather ⓘ
all-reduce ⓘ
broadcast ⓘ
gather ⓘ
reduce ⓘ
reduce-scatter ⓘ
scatter ⓘ
supportsReductionOp max ⓘ
min ⓘ
product ⓘ
sum ⓘ
supportsTopology multi-node multi-GPU ⓘ
single-node multi-GPU ⓘ
targetUser HPC developers ⓘ
deep learning framework developers ⓘ
usedBy DeepSpeed ⓘ
Horovod ⓘ
Megatron-LM ⓘ
PyTorch distributed ⓘ
linked to: PyTorch

TensorFlow distributed training ⓘ

How these facts were elicited

Referenced by (12)

Full triples — surface form annotated when it differs from this entity's canonical label.

NVIDIA CUDA → includes → NCCL ⓘ
NVIDIA Tesla data center GPUs → supportsEcosystem → NVIDIA NCCL ⓘ
linked to: NCCL
NCCL → fullName → NVIDIA Collective Communications Library ⓘ
linked to: NCCL
CUDA → includesLibrary → NCCL ⓘ
NVIDIA HPC SDK → hasComponent → NVIDIA Communication Libraries ⓘ
linked to: NCCL
Megatron-LM → uses → NVIDIA NCCL ⓘ
linked to: NCCL
Hopper → supportsParallelProgrammingModel → NVIDIA NCCL ⓘ
linked to: NCCL
NVIDIA A30 → compatibleWith → NVIDIA NCCL ⓘ
linked to: NCCL