Packet-level clustering for memory-assisted compression of network packets

With the explosive growth of the Internet traffic, data compression can be a powerful technique to improve the efficiency of data transfer in networks and consequently reduce the cost associated with the transmission of such data. Recently, we proposed a memory-assisted compression framework that utilizes the packet-level memorized context to reduce the inevitable redundancy in the universal compression of the payloads of the short-length network packets. In this paper, we investigate the practical aspects of implementing cluster-based memory-assisted compression and proposed a non-parametric clustering algorithm for training packet selection. We demonstrate that, when compression speed is not an issue, our proposed non-parametric clustering algorithm with Lite PAQ compression algorithm can achieve nearly 70% traffic reduction on real data gathered from Internet traffic. We also explore the trade-off between the memory-assisted compression speed and performance using different clustering algorithms and compression methods.