Engineering a Lightweight External Memory Suffix Array Construction Algorithm

• Published in: Mathematics in computer science Vol. 11; no. 2; pp. 137 - 149
• Main Authors: Kärkkäinen, Juha, Kempa, Dominik
• Format: Journal Article
• Language: English
• Published: Cham Springer International Publishing 01.06.2017; Springer Nature B.V
• Subjects: Algorithms; Computer Science; Mathematics; Mathematics and Statistics; Random access memory; 68W32 (Algorithms on strings); External memory algorithms; Algorithm engineering; Suffix array construction
• ISSN: 1661-8270; 1661-8289
• DOI: 10.1007/s11786-016-0281-1

We describe an external memory suffix array construction algorithm based on constructing suffix arrays for blocks of text and merging them into the full suffix array. The basic idea goes back over 20 years and there has been a couple of later improvements, but we describe several further improvements that make the algorithm much faster. In particular, we reduce the I/O volume of the algorithm by a factor O log σ n . Our experiments show that the algorithm is the fastest suffix array construction algorithm when the size of the text is within a factor of about five from the size of the RAM in either direction, which is a common situation in practice.