An Improved Two-stream 3D Convolutional Neural Network for Human Action Recognition

Jun Chen, Yuanping Xu, Chaolong Zhang, Zhijie Xu, Xiangxiang Meng, Jie Wang

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

In order to obtain global contextual information precisely from videos with heavy camera motions and scene changes, this study proposes an improved spatiotemporal two-stream neural network architecture with a novel convolutional fusion layer. The three main improvements of this study are: 1) the Resnet-101 network has been integrated into the two streams of the target network independently; 2) two kinds of feature maps (i.e., the optical flow motion and RGB-channel information) obtained by the corresponding convolution layer of two streams respectively are superimposed on each other; 3) the temporal information is combined with the spatial information by the integrated three-dimensional (3D) convolutional neural network (CNN) to extract more latent information from the videos. The proposed approach was tested by using UCF-101 and HMDB51 benchmarking datasets and the experimental results show that the proposed two-stream 3D CNN model can gain substantial improvement on the recognition rate in video-based analysis.

Original languageEnglish
Title of host publication2019 25th IEEE International Conference on Automation and Computing
Subtitle of host publicationImproving Productivity through Automation and Computing
EditorsHui Yu
PublisherIEEE
Number of pages6
ISBN (Electronic)9781861376657, 9781861376664
ISBN (Print)9781728125183
DOIs
Publication statusPublished - 11 Nov 2019
Event25th IEEE International Conference on Automation and Computing: Improving Productivity through Automation and Computing - Lancaster University, Lancaster, United Kingdom
Duration: 5 Sep 20197 Sep 2019
Conference number: 25
http://www.research.lancs.ac.uk/portal/en/activities/25th-ieee-international-conference-on-automation-and-computing-icac19-57-september-2019-lancaster-university-uk(679d94ff-4efb-46b5-9c80-c6d34a13bae4).html
http://www.research.lancs.ac.uk/portal/en/activities/general-conference-chair-of-the-25th-ieee-international-conference-on-automation-and-computing-icac19-57-september-2019-lancaster-university-uk(ea0392e6-0187-4468-a934-5f1aa49479de).html

Conference

Conference25th IEEE International Conference on Automation and Computing
Abbreviated titleICAC 2019
CountryUnited Kingdom
CityLancaster
Period5/09/197/09/19
Internet address

    Fingerprint

Cite this

Chen, J., Xu, Y., Zhang, C., Xu, Z., Meng, X., & Wang, J. (2019). An Improved Two-stream 3D Convolutional Neural Network for Human Action Recognition. In H. Yu (Ed.), 2019 25th IEEE International Conference on Automation and Computing: Improving Productivity through Automation and Computing [8894962] IEEE. https://doi.org/10.23919/IConAC.2019.8894962