Evaluating window joins over unbounded streams

• Published in: 2003 19th International Conference on Data Engineering pp. 341 - 352
• Main Authors: Kang, J., Naughton, J.F., Viglas, S.D.
• Format: Conference Proceeding
• Language: English
• Published: IEEE 2003
• Subjects: Data engineering
• ISBN: 9780780376656; 078037665X
• DOI: 10.1109/ICDE.2003.1260804

We investigate algorithms for evaluating sliding window joins over pairs of unbounded streams. We introduce a unit-time-basis cost model to analyze the expected performance of these algorithms. Using this cost model, we propose strategies for maximizing the efficiency of processing joins in three scenarios. First, we consider the case where one stream is much faster than the other. We show that asymmetric combinations of join algorithms, (e.g., hash join on one input, nested-loops join on the other) can outperform symmetric join algorithm implementations. Second, we investigate the case where system resources are insufficient to keep up with the input streams. We show that we can maximize the number of join result tuples produced in this case by properly allocating computing resources across the two input streams. Finally, we investigate strategies for maximizing the number of result tuples produced when memory is limited, and show that proper memory allocation across the two input streams can result in significantly lower resource usage and/or more result tuples produced.