InfiniBand In-Network Computing Technology and Roadmap

Contributors:

• Paul Calleja (University Of Cambridge)
• Daniel Gruner (University of Toronto)
• Dhabaleswar Panda (Ohio State University)
• Gilad Shainer (NVIDIA)

---

Abstract:

The past focus for smart interconnects development was to offload the network functions from the CPU to the network. With the new efforts in the co-design approach, the new generation of smart interconnects also offload data algorithms that are managed within the network, allowing users to run these algorithms as the data being transferred within the system interconnect, rather than waiting for the data to reach the CPU. This technology is being referred to as In-Network Computing or IO Processing Unites (IPUs). In-Network Computing transforms the data center interconnect to become a “distributed CPU, enables to overcome performance walls and to enable faster and more scalable data analysis.

HDR 200G InfiniBand In-Network Computing technology includes several elements - Scalable Hierarchical Aggregation and Reduction Protocol (SHARP), a technology that was developed by Oak Ridge National Laboratory and Mellanox and received the R&D100 award, smart Tag Matching and rendezvoused protocol, and more. These technologies are in use at some of the recent large-scale supercomputers around the world, including the top TOP500 platforms.

The session will discuss the latest development around InfiniBand In-Network Computing technology and testing results from DoE systems, Canada’s fastest InfiniBand Dragonfly based supercomputer at the University of Toronto, TACC Frontera supercomputer, and other HDR InfiniBand supercomputers. It will also covers the integration of In-Network Computing into various programming models.

As the needs for faster data speed accelerates, the InfiniBand Trade Association has been working to set the goals for future speeds, and this topic will also be covered.