Predicting sporadic grid data transfers

The increasingly common practice of replicating datasets and using resources as distributed data stores in grid environments has led to the problem of determining which replica can be accessed most efficiently. Due diverse performance characteristics and load variations of several components in the...

Full description

Saved in:
Bibliographic Details
Published in11th International Symposium on High-Performance Distributed Computing (HPDC-11 2002) pp. 188 - 196
Main Authors Vazhkudai, S., Schopf, J.M.
Format Conference Proceeding
LanguageEnglish
Published IEEE 2002
Subjects
Online AccessGet full text
ISBN0769516866
9780769516868
ISSN1082-8907
DOI10.1109/HPDC.2002.1029918

Cover

More Information
Summary:The increasingly common practice of replicating datasets and using resources as distributed data stores in grid environments has led to the problem of determining which replica can be accessed most efficiently. Due diverse performance characteristics and load variations of several components in the end-to-end path linking these various locations, selecting a replica from among many requires accurate prediction information of the data transfer times between the sources and sinks. In this paper we present a prediction system that is based on combining end-to-end application throughput observations and network load variations, capturing the whole-system performance and variations in load patterns, respectively. We develop a set of regression models to derive predictions that characterize the effect of network load variations on file transfer times. We apply these techniques to the GridFTP data movement tool, part of the Globus Toolkit/spl trade/, and observe performance gains of up to 10% in prediction accuracy when compared with approaches based on past system behavior in isolation.
Bibliography:SourceType-Conference Papers & Proceedings-1
ObjectType-Conference Paper-1
content type line 25
ISBN:0769516866
9780769516868
ISSN:1082-8907
DOI:10.1109/HPDC.2002.1029918