Deep neural network based single pixel prediction for unified video coding

• Published in: Neurocomputing (Amsterdam) Vol. 272; pp. 558 - 570
• Main Authors: Li, Honggui, Trocan, Maria
• Format: Journal Article
• Language: English
• Published: Elsevier B.V 10.01.2018
• Subjects: Deep neural network; Inter-frame coding; Intra-frame coding; Multi-view coding; Unified video coding; Video prediction; Multi-view coding; Deep neural network; Intra-frame coding; Video prediction; Inter-frame coding; Unified video coding
• ISSN: 0925-2312; 1872-8286
• DOI: 10.1016/j.neucom.2017.07.037

Classical video prediction methods exploit directly and shallowly the intra-frame, inter-frame and multi-view similarities within the video sequences; the proposed video prediction methods indirectly and intensively transform the frame correlations into nonlinear mappings by using a general deep neural network (DNN) with single output node. Traditional DNN based video prediction algorithms wholly and coarsely forecast the next frame, but the proposed video prediction algorithms severally and precisely anticipate single pixel of future frame in order to achieve high prediction accuracy and low computation cost. First of all, general DNN based prediction algorithms for intra-frame coding, inter-frame coding and multi-view coding are presented respectively. Then, general DNN based prediction algorithm for unified video coding is raised, which relies on the preceding three prediction algorithms. It is evaluated by simulation experiments that the proposed methods hold better performance than state of the art High Efficiency Video Coding (HEVC) in peak signal to noise ratio (PSNR) and bit per pixel (BPP) in the situation of low bitrate transmission. It is also verified by experimental results that the proposed general DNN architecture possesses higher prediction accuracy and lower computation load than those of conventional DNN architectures. It is further testified by experimental results that the proposed methods are very suitable for multi-view videos with small correlations and big disparities.