The advent of RoCE (RDMA over Converged Ethernet) has led to a significant
increase in the use of RDMA in datacenter networks. To achieve good
performance, RoCE requires a lossless network which is in turn achieved by
enabling Priority Flow Control (PFC) within the network. However, PFC brings
with it a host of problems such as head-of-the-line blocking, congestion
spreading, and occasional deadlocks. Rather than seek to fix these issues, we
instead ask: is PFC fundamentally required to support RDMA over Ethernet?
We show that the need for PFC is an artifact of current RoCE NIC designs
rather than a fundamental requirement. We propose an improved RoCE NIC (IRN)
design that makes a few simple changes to the RoCE NIC for better handling of
packet losses. We show that IRN (without PFC) outperforms RoCE (with PFC) by
6-83% for typical network scenarios. Thus not only does IRN eliminate the need
for PFC, it improves performance in the process! We further show that the
changes that IRN introduces can be implemented with modest overheads of about
3-10% to NIC resources. Based on our results, we argue that research and
industry should rethink the current trajectory of network support for RDMA.