Dual-channel CNN for efficient abnormal behavior identification through crowd feature engineering

Yuanping Xu, Li Lu, Zhijie Xu, Jia He, Jiliu Zhou, Chaolong Zhang

Research output: Contribution to journalArticle

1 Citation (Scopus)

Abstract

This research has been investigating an automatic and online crowd anomaly detection model by exploring a novel compound image descriptor generated from live video streams. A dual-channel convolutional neural network (DCCNN) has been set up for efficiently processing scene-related and motion-related crowd information inherited from raw frames and the compound descriptor instances. The novelty of the work stemmed from the creation of the spatio-temporal cuboids in online (or near real-time) manner through dynamically extracting local feature tracklets within the temporal space while handling the foreground region-of-interests (i.e., moving targets) through the exploration of Gaussian Mixture Model in the spatial space. Hence, the extracted foreground blocks can effectively eliminate irrelevant backgrounds and noises from the live streams for reducing the computational costs in the subsequent detecting phases. The devised compound feature descriptor, named as spatio-temporal feature descriptor (STFD), is capable of characterizing the crowd attributes through the measures such as collectiveness, stability, conflict and density in each online generated spatio-temporal cuboid. A STFD instance registers not only the dynamic variation of the targeted crowd over time based on local feature tracklets, but also the interaction information of neighborhoods within a crowd, e.g., the interaction force through the K-nearest neighbor (K-NN) analysis. The DCCNN developed in this research enables online identification of suspicious crowd behaviors based on analyzing the live-feed images and their STFD instances. The proposed model has been developed and evaluated against benchmarking techniques and databases. Experimental results have shown substantial improvements in terms of detection accuracy and efficiency for online crowd abnormal behavior identification.
Original languageEnglish
Pages (from-to)945-958
Number of pages14
JournalMachine Vision and Applications
Volume30
Issue number5
Early online date25 Aug 2018
DOIs
Publication statusPublished - Jul 2019

Fingerprint

Neural networks
Benchmarking
Processing
Costs

Cite this

Xu, Yuanping ; Lu, Li ; Xu, Zhijie ; He, Jia ; Zhou, Jiliu ; Zhang, Chaolong. / Dual-channel CNN for efficient abnormal behavior identification through crowd feature engineering. In: Machine Vision and Applications. 2019 ; Vol. 30, No. 5. pp. 945-958.
@article{8830f917d0994ffc9aaa06f3b8723312,
title = "Dual-channel CNN for efficient abnormal behavior identification through crowd feature engineering",
abstract = "This research has been investigating an automatic and online crowd anomaly detection model by exploring a novel compound image descriptor generated from live video streams. A dual-channel convolutional neural network (DCCNN) has been set up for efficiently processing scene-related and motion-related crowd information inherited from raw frames and the compound descriptor instances. The novelty of the work stemmed from the creation of the spatio-temporal cuboids in online (or near real-time) manner through dynamically extracting local feature tracklets within the temporal space while handling the foreground region-of-interests (i.e., moving targets) through the exploration of Gaussian Mixture Model in the spatial space. Hence, the extracted foreground blocks can effectively eliminate irrelevant backgrounds and noises from the live streams for reducing the computational costs in the subsequent detecting phases. The devised compound feature descriptor, named as spatio-temporal feature descriptor (STFD), is capable of characterizing the crowd attributes through the measures such as collectiveness, stability, conflict and density in each online generated spatio-temporal cuboid. A STFD instance registers not only the dynamic variation of the targeted crowd over time based on local feature tracklets, but also the interaction information of neighborhoods within a crowd, e.g., the interaction force through the K-nearest neighbor (K-NN) analysis. The DCCNN developed in this research enables online identification of suspicious crowd behaviors based on analyzing the live-feed images and their STFD instances. The proposed model has been developed and evaluated against benchmarking techniques and databases. Experimental results have shown substantial improvements in terms of detection accuracy and efficiency for online crowd abnormal behavior identification.",
keywords = "Crowd abnormal behaviors, Dual-channel convolutional neural network, Gaussian Mixture Model, Spatio-temporal feature descriptor, Crowd feature engineering",
author = "Yuanping Xu and Li Lu and Zhijie Xu and Jia He and Jiliu Zhou and Chaolong Zhang",
note = "This is a post-peer-review, pre-copyedit version of an article published in Machine Vision and Applications. The final authenticated version is available online at: https://link.springer.com/article/10.1007{\%}2Fs00138-018-0971-6",
year = "2019",
month = "7",
doi = "10.1007/s00138-018-0971-6",
language = "English",
volume = "30",
pages = "945--958",
journal = "Machine Vision and Applications",
issn = "0932-8092",
publisher = "Springer Verlag",
number = "5",

}

Dual-channel CNN for efficient abnormal behavior identification through crowd feature engineering. / Xu, Yuanping; Lu, Li; Xu, Zhijie; He, Jia; Zhou, Jiliu; Zhang, Chaolong.

In: Machine Vision and Applications, Vol. 30, No. 5, 07.2019, p. 945-958.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Dual-channel CNN for efficient abnormal behavior identification through crowd feature engineering

AU - Xu, Yuanping

AU - Lu, Li

AU - Xu, Zhijie

AU - He, Jia

AU - Zhou, Jiliu

AU - Zhang, Chaolong

N1 - This is a post-peer-review, pre-copyedit version of an article published in Machine Vision and Applications. The final authenticated version is available online at: https://link.springer.com/article/10.1007%2Fs00138-018-0971-6

PY - 2019/7

Y1 - 2019/7

N2 - This research has been investigating an automatic and online crowd anomaly detection model by exploring a novel compound image descriptor generated from live video streams. A dual-channel convolutional neural network (DCCNN) has been set up for efficiently processing scene-related and motion-related crowd information inherited from raw frames and the compound descriptor instances. The novelty of the work stemmed from the creation of the spatio-temporal cuboids in online (or near real-time) manner through dynamically extracting local feature tracklets within the temporal space while handling the foreground region-of-interests (i.e., moving targets) through the exploration of Gaussian Mixture Model in the spatial space. Hence, the extracted foreground blocks can effectively eliminate irrelevant backgrounds and noises from the live streams for reducing the computational costs in the subsequent detecting phases. The devised compound feature descriptor, named as spatio-temporal feature descriptor (STFD), is capable of characterizing the crowd attributes through the measures such as collectiveness, stability, conflict and density in each online generated spatio-temporal cuboid. A STFD instance registers not only the dynamic variation of the targeted crowd over time based on local feature tracklets, but also the interaction information of neighborhoods within a crowd, e.g., the interaction force through the K-nearest neighbor (K-NN) analysis. The DCCNN developed in this research enables online identification of suspicious crowd behaviors based on analyzing the live-feed images and their STFD instances. The proposed model has been developed and evaluated against benchmarking techniques and databases. Experimental results have shown substantial improvements in terms of detection accuracy and efficiency for online crowd abnormal behavior identification.

AB - This research has been investigating an automatic and online crowd anomaly detection model by exploring a novel compound image descriptor generated from live video streams. A dual-channel convolutional neural network (DCCNN) has been set up for efficiently processing scene-related and motion-related crowd information inherited from raw frames and the compound descriptor instances. The novelty of the work stemmed from the creation of the spatio-temporal cuboids in online (or near real-time) manner through dynamically extracting local feature tracklets within the temporal space while handling the foreground region-of-interests (i.e., moving targets) through the exploration of Gaussian Mixture Model in the spatial space. Hence, the extracted foreground blocks can effectively eliminate irrelevant backgrounds and noises from the live streams for reducing the computational costs in the subsequent detecting phases. The devised compound feature descriptor, named as spatio-temporal feature descriptor (STFD), is capable of characterizing the crowd attributes through the measures such as collectiveness, stability, conflict and density in each online generated spatio-temporal cuboid. A STFD instance registers not only the dynamic variation of the targeted crowd over time based on local feature tracklets, but also the interaction information of neighborhoods within a crowd, e.g., the interaction force through the K-nearest neighbor (K-NN) analysis. The DCCNN developed in this research enables online identification of suspicious crowd behaviors based on analyzing the live-feed images and their STFD instances. The proposed model has been developed and evaluated against benchmarking techniques and databases. Experimental results have shown substantial improvements in terms of detection accuracy and efficiency for online crowd abnormal behavior identification.

KW - Crowd abnormal behaviors

KW - Dual-channel convolutional neural network

KW - Gaussian Mixture Model

KW - Spatio-temporal feature descriptor

KW - Crowd feature engineering

UR - http://www.scopus.com/inward/record.url?scp=85053041397&partnerID=8YFLogxK

U2 - 10.1007/s00138-018-0971-6

DO - 10.1007/s00138-018-0971-6

M3 - Article

VL - 30

SP - 945

EP - 958

JO - Machine Vision and Applications

JF - Machine Vision and Applications

SN - 0932-8092

IS - 5

ER -