Properties and prediction of flow statistics from sampled packet streams

Abstract

Many routers can generate and export statistics on flows of packets that traverse them. Increasingly, high end routers form flow statistics from only a sampled packet stream in order to manage resource consumption involved.This paper addresses three questions. Firstly: what are the downstream consequences for the measurement infrastructure? Long traffic flows will be split up if the time between sampled packets exceeds the flow timeout. Using packet header traces we show that flows generated by increasingly prevalent peer-to-peer applicalions are vulnerable to this effect.Secondly: can the volume of packet-sampled flow statistics be easily determined? We develop a simple model that predicts both the export rate of flow packet-sampled flow statistics and the number of active flows. It uses unsampled flow statistics---those commonly currently collected--as its data, i.e., it does not rely on having packet header traces available.Thirdly: what properties of the original traffic stream can be inferred from the packet sampled flow statistics? We show that as well as estimating total bytes and packets, one can also infer more detail, specifically the number and average length of flows in the unsampled traffic stream, even though some flows will have no packets sampled. We believe that this information is useful, both for understanding source traffic, e.g. the dependence of flow lengths on application type, and also monitoring changes in the composition of the traffic, e.g., a flood of short flows during a DoS attack. In all cases, we evaluate our approach using packet header traces gathered in backbone and campus networks.