GPU Direct RDMA is Finally Out!

GPUDirect RDMA is the newest technology for GPU to GPU communications over the InfiniBand interconnect. GPUDirect RDMA enables a direct data transfer from the GPU memory over the InfiniBand network via PCI Pier-to-Pier (P2P). This capability introduced in the NVIDIA Kepler-class GPUs, CUDA 5.0 and the Mellanox InfiniBand solutions.

The importance of this capability is with bypassing the CPU for GPU communications (who needs the CPU…..), therefore a dramatic increase in performance. Finally after long time of waiting, the two companies mentioned above have demonstrated the new capability in the recent ISC’13 conference. Prof. Dhabaleswar K. (DK) Panda, Hari Subramoni and Sreeram Potluri from the Ohio State University presented at the HPC Advisory Council their first results with the GPU Direct RDMA – 70% reduction in latency! You can see the entire presentation at http://www.hpcadvisorycouncil.com/events/2013/European-Workshop/presentations/9_OSU.pdf. Seems that GE Intelligent Platforms already using the new technology - http://www.militaryaerospace.com/whitepapers/2013/03/gpudirect_-rdma.html, which is a great example of how the new capability can make our life better (or faster…). You can also read more on http://docs.nvidia.com/cuda/gpudirect-rdma/index.html.



In the graph: latency improvement presented by DK Panda