Mahout: Low-overhead datacenter traffic management using end-host-based elephant detection

Abstract

Datacenters need high-bandwidth interconnection fabrics. Several researchers have proposed highly-redundant topologies with multiple paths between pairs of end hosts for datacenter networks. However, traffic management is necessary to effectively utilize the bisection bandwidth provided by these topologies. This requires timely detection of elephant flows-flows that carry large amount of data-and managing those flows. Previously proposed approaches incur high monitoring overheads, consume significant switch resources, and/or have long detection times. We propose, instead, to detect elephant flows at the end hosts. We do this by observing the end hosts's socket buffers, which provide better, more efficient visibility of flow behavior. We present Mahout, a low-overhead yet effective traffic management system that follows OpenFlow-like central controller approach for network management but augments the design with our novel end host mechanism. Once an elephant flow is detected, an end host signals the network controller using in-band signaling with low overheads. Through analytical evaluation and experiments, we demonstrate the benefits of Mahout over previous solutions.