Order Matters: Shuffling Sequence Generation for Video Prediction

Junyan Wang (Newcastle University), BingZhang Hu (Newcastle University), Yang Long (Newcastle University), Yu Guan (Newcastle University)

Abstract
Predicting future frames in natural video sequences is a new challenge that is receiving increasing attention in the computer vision community. However, existing models suffer from severe loss of temporal information when the predicted sequence is long. Compared to previous methods focusing on generating more realistic contents, this paper extensively studies the importance of sequential order information for video generation. A novel Shuffling sEquence gEneration network (SEE-Net) is proposed that can learn to discriminate between natural and unnatural sequential orders by shuffling the video frames and comparing them to the real video sequences. Systematic experiments on three datasets with both synthetic and real-world videos manifest the effectiveness of shuffling sequence generation for video prediction in our proposed model and demonstrate state-of-the-art performance by both qualitative and quantitative evaluations. The source code is available at \url{https://github.com/andrewjywang/SEENet}.

DOI
10.5244/C.33.202
https://dx.doi.org/10.5244/C.33.202

Files
Paper (PDF)

BibTeX