A Spatiotemporal Pre-processing Network for Activity Recognition under Rain

Minah Lee (Georgia Institute of Technology), Burhan Mudassar (Georgia Institute of Technology), Taesik Na (Georgia Institute of Technology), Saibal Mukhopadhyay (Georgia Institute of Technology)

Abstract
This paper presents a deep neural network (DNN) based fully spatiotemporal rain removal network, MoPE-Spatiotemporal, to enhance accuracy of activity recognition in rainy videos. The proposed network utilizes spatiotemporal information of an image sequence to detect rain streaks and recover the non-rainy image. We also present rain alert network that detects the rain fall and informs the reduction of recognition confidence under rain. Experimental results show that heavy rain can highly degrade activity recognition accuracy. MoPE-Spatiotemporal removes heavy rain better than state-of-the-art methods, and significantly improves (0.15) activity recognition accuracy in rainy videos with minimal impact on recognition accuracy in clean videos.

DOI
10.5244/C.33.172
https://dx.doi.org/10.5244/C.33.172

Files
Paper (PDF)

BibTeX