Double Index NEsted-Loop Reactive Join for Result Rate Optimization

• Published in: 2009 IEEE 25th International Conference on Data Engineering pp. 481 - 492
• Main Authors: Bornea, M.A., Vassalos, V., Kotidis, Y., Deligiannakis, A.
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01.03.2009
• Subjects: Application software; Computer networks; Computer science; Data engineering; Delay; Pipeline processing; Production; Query processing; Smoothing methods; Switches
• ISBN: 9781424434220; 142443422X
• ISSN: 1063-6382
• DOI: 10.1109/ICDE.2009.101

Adaptive join algorithms have recently attracted a lot of attention in emerging applications where data is provided by autonomous data sources through heterogeneous network environments. Their main advantage over traditional join techniques is that they can start producing join results as soon as the first input tuples are available, thus improving pipelining by smoothing join result production and by masking source or network delays. In this paper we propose double index nested loops reactive join (DINER), a new adaptive join algorithm for result rate maximization. DINER combines two key elements: an intuitive flushing policy that aims to increase the productivity of in-memory tuples in producing results during the online phase of the join, and a novel re-entrant join technique that allows the algorithm to rapidly switch between processing in-memory and disk-resident tuples, thus better exploiting temporary delays when new data is not available. Our experiments using real and synthetic data sets demonstrate that DINER outperforms previous adaptive join algorithms in producing result tuples at a significantly higher rate, while making better use of the available memory.