Improving I/O Forwarding Throughput with Data Compression

While network bandwidth is steadily increasing, it is doing so at a much slower rate than the corresponding increase in CPU performance. This trend has widened the gap between CPU and network speed. In this paper, we investigate improvements to I/O performance by exploiting this gap. We harness idle...

Full description

Saved in:
Bibliographic Details
Published in2011 IEEE International Conference on Cluster Computing pp. 438 - 445
Main Authors Welton, B., Kimpe, D., Cope, J., Patrick, C. M., Iskra, K., Ross, R.
Format Conference Proceeding
LanguageEnglish
Published IEEE 01.09.2011
Subjects
Bandwidth
Benchmark testing
Data compression
I/O Forwarding
Libraries
Random access memory
Scientific Data Compression
Servers
Throughput
Online AccessGet full text
ISBN9781457713552
1457713551
ISSN1552-5244
DOI10.1109/CLUSTER.2011.80

Cover

More Information
Summary:While network bandwidth is steadily increasing, it is doing so at a much slower rate than the corresponding increase in CPU performance. This trend has widened the gap between CPU and network speed. In this paper, we investigate improvements to I/O performance by exploiting this gap. We harness idle CPU resources to compress network traffic, reducing the amount of data transferred over the network and increasing effective network bandwidth. We created a set of compression services within the I/O Forwarding Scalability Layer. These services transparently compress and decompress data as it is transferred over the network. We studied the effect of the compression services on a variety of data sets and conducted experiments on a high-performance computing cluster.
ISBN:9781457713552
1457713551
ISSN:1552-5244
DOI:10.1109/CLUSTER.2011.80