zipHMMlib: a highly optimised HMM library exploiting repetitions in the input to speed up the forward algorithm

• Published in: BMC bioinformatics Vol. 14; no. 1; p. 339
• Main Authors: Sand, Andreas, Kristiansen, Martin, Pedersen, Christian NS, Mailund, Thomas
• Format: Journal Article
• Language: English
• Published: London BioMed Central 22.11.2013; BioMed Central Ltd; Springer Nature B.V
• Subjects: Algorithms; Analysis; Animals; Bioinformatics; Biomedical and Life Sciences; Computation; Computational Biology/Bioinformatics; Computer Appl. in Life Sciences; Computer programs; Computer Simulation; Construction; Gorilla gorilla - genetics; Humans; Libraries; Life Sciences; Likelihood Functions; Markov analysis; Markov Chains; Markov processes; Mathematical models; Microarrays; Observational Studies as Topic; Pan troglodytes - genetics; Peptide Library; Phylogeny; Pongo - genetics; Preprocessing; Probability; Science; Sequence analysis (methods); Software; Time Factors; Alphabet Size; Forward Algorithm; Hide State; Input Sequence; Hide Markov Model
• ISSN: 1471-2105; 1471-2105
• DOI: 10.1186/1471-2105-14-339

Background Hidden Markov models are widely used for genome analysis as they combine ease of modelling with efficient analysis algorithms. Calculating the likelihood of a model using the forward algorithm has worst case time complexity linear in the length of the sequence and quadratic in the number of states in the model. For genome analysis, however, the length runs to millions or billions of observations, and when maximising the likelihood hundreds of evaluations are often needed. A time efficient forward algorithm is therefore a key ingredient in an efficient hidden Markov model library. Results We have built a software library for efficiently computing the likelihood of a hidden Markov model. The library exploits commonly occurring substrings in the input to reuse computations in the forward algorithm. In a pre-processing step our library identifies common substrings and builds a structure over the computations in the forward algorithm which can be reused. This analysis can be saved between uses of the library and is independent of concrete hidden Markov models so one preprocessing can be used to run a number of different models. Using this library, we achieve up to 78 times shorter wall-clock time for realistic whole-genome analyses with a real and reasonably complex hidden Markov model. In one particular case the analysis was performed in less than 8 minutes compared to 9.6 hours for the previously fastest library. Conclusions We have implemented the preprocessing procedure and forward algorithm as a C++ library, zipHMM, with Python bindings for use in scripts. The library is available at http://birc.au.dk/software/ziphmm/ .