Parallel computing method of deep belief networks and its application to traffic flow prediction

• Published in: Knowledge-Based Systems Vol. 163; pp. 972 - 987
• Main Authors: Zhao, Lu, Zhou, Yonghua, Lu, Huapu, Fujita, Hamido
• Format: Journal Article
• Language: English; Japanese
• Published: Amsterdam Elsevier B.V 01.01.2019; Elsevier BV; Elsevier Science Ltd
• Subjects: Algorithms; Belief networks; Computation; Deep belief network; Deep learning; Image processing; Learning; Machine learning; Natural language processing; Nodes; Object recognition; Parallel computing; Predictions; Speech recognition; Synchronism; Traffic flow; Traffic flow prediction; Training; Deep learning; Traffic flow prediction; Parallel computing; Deep belief network
• ISSN: 0950-7051; 1872-7409
• DOI: 10.1016/j.knosys.2018.10.025

Deep belief networks (DBNs) with outstanding advantages of learning input data features have attained particular attention and are applied widely in image processing, speech recognition, natural language interpretation, disease diagnosis, among others. However, owing to large data, the training processes of DBNs are time-consuming and may not satisfy the requirements of real-time application systems. In this study, a single dataset is decomposed into multiple subdatasets that are distributed to multiple computing nodes. Multiple computing nodes learn the features of their own subdatasets. On the precondition of the remaining features where one computing node learns from the total dataset, the single dataset learning models and algorithms are extended to the cases where multiple computing nodes learn multiple subdatasets in a parallel manner. Learning models and algorithms are proposed for the parallel computing of DBN learning processes. A master–slave parallel computing structure is designed, where the slave computing nodes learn the features of their respective subdatasets and transmit them to the master computing node. The master computing node is critical in synthesizing the learned features from the respective slave computing nodes. The broadcast, synchronization, and synthesis are repeated until all features of subdatasets have been learned. The proposed parallel computing method is applied to traffic flow prediction using practical traffic flow data. Our experimental results verify the effectiveness of the parallel computing method of DBN learning processes in terms of decreasing pre-training and fine-tuning times and maintaining the prominent feature learning abilities. •A parallel computing method for the DBN pre-training and fine-tuning phases is proposed.•The parallel computing method is based on the master–slave and data parallel processing structure.•The parallel computing method is applied to traffic flow prediction.•Experimental results testify the effectiveness of the parallel computing method.•The performances are compared between the serial and parallel computing methods.