Challenges in Evaluating New Data Center Network Protocols

With the rapid development of computer systems and the ever-growing amounts of data that modern applications process, the requirements for data centers are also constantly evolving. The growing communication demands require operators to rethink their network infrastructure, like network hardware components or network protocols.

For example, the network protocol Homa takes a new approach and focuses on connectionless, message-oriented remote procedure calls (RPCs) as opposed to the widely used connection-oriented TCP protocol. Such new protocols can outperform established protocols with new approaches. However, there are substantial hurdles in adopting them, as this requires extensive testing and often intrusive changes in the network infrastructure. In this post, we discuss these challenges of testing and adopting a new network protocol in a data center, using Homa as an example. We will also touch on how virtual prototyping can help overcome these challenges.

The Homa Network Protocol

Despite the increasing amount of data that is exchanged in a data center, a non-negligible part of common workloads consists of small messages. Today’s standard transport protocols such as TCP do not focus on small messages but instead on large ones, which results in poor performance for small messages, especially high and inconsistent latency. With our standard connection-based sequential protocols, small messages can experience head-of-line blocking, because they get stuck behind larger messages in the flow. To improve tail latencies for small messages, Homa approximates SRPT (shortest remaining processing time first) through a receiver-driven approach, i.e. the receiver informs the sender which message to prioritize when sending data. Homa is connectionless, message-based, and uses priority queues provided by modern network switches.

Challenges Of Adopting Homa

In contrast to today’s standard protocols, Homa makes use of special features of network switches. As a result, adopting it in a data center requires substantial changes to network hardware and software. The message-based nature of the protocol dictates a new API that focuses on a request-response pattern typical for RPCs, which requires adapting the applications to use Homa. Further, Homa requires changes to all network switches to configure a specific priority queuing configuration. Additionally, operators need to install the Homa Linux kernel module on each host. Consequently, deploying Homa in a data center is complicated and requires intrusive changes, possibly even buying and installing new hardware.

Evaluating Homa For a Data Center

Homa comes with a variety of configuration options that let the user tune the protocol to the workload and specific setup like the available link bandwidth. This gives the user a large configuration space to explore. Additionally, the user has to test application integrations and the resulting performance benefits. Due to the complexity of integrating Homa in a data center, it is difficult to test the protocol at scale. Even smaller testbeds are time-consuming to configure, especially when exploring different configuration options.

Virtual Prototyping For Testing And Developing

Virtual prototyping can help overcome these challenges by allowing data center engineers to create a virtual replica of the data center. This replica serves as a testbench to test and evaluate the performance of Homa before implementing it in the actual network. The virtual replica makes it easy to swap and rearrange components and adjust configuration options. Additionally, the data center engineer can scale the virtual prototype from a small testbed to a large-scale system using distributed simulations. Finally, software developers can use the virtual prototype for adapting and testing applications with the new network protocol, ensuring seamless integration and optimal performance.

If you have any questions or would like to learn more, please do no hesitate to reach out: