Towards Intelligent Crowd Behavior Understanding through the STFD Descriptor Exploration

Yuanping Xu, Li Lu, Zhijie Xu, Jia He, Jing Wang, Jian Huang, Jun Lu

Research output: Contribution to journalArticle

1 Citation (Scopus)

Abstract

Realizing the automated and online detection of crowd anomalies from surveillance CCTVs is a research-intensive and application-demanding task. This research proposes a novel technique for detecting crowd abnormalities through analyzing the spatial and temporal features of input video signals. This integrated solution defines an image descriptor (named spatio-temporal feature descriptor—STFD) that reflects the global motion pattern of crowds over time. A designed convolutional neural network (CNN) has then been adopted to classify dominant or large-scale crowd abnormal behaviors. The work reported has focused on: (1) detecting moving objects in online (or near real-time) manner through spatio-temporal segmentations of crowds identified by the similarity of group trajectory structures in the temporal space and the foreground blocks based on the Gaussian mixture model in the spatial space; (2) dividing multiple clustered groups based on the spectral clustering methods through treating image pixels from segmented regions as dynamic particles; (3) creating STFD descriptor instances by calculating corresponding attributes such as collectiveness, stability, conflict and crowd density for individuals (particles) in the corresponding groups; (4) inputting generated STFD descriptor instances into the devised CNN to detect suspicious crowd behaviors. For the test and evaluation of the devised models and techniques, the PETS database has been selected as the primary experimental data sets. Results against benchmarking models and systems have shown promising advancements of this novel approach in terms of accuracy and efficiency for crowd anomaly detection.
Original languageEnglish
Article number17
Pages (from-to)1-22
Number of pages22
JournalSensing and Imaging
Volume19
Issue number1
Early online date25 Apr 2018
DOIs
Publication statusPublished - Dec 2018

Fingerprint

anomalies
Closed circuit television systems
Neural networks
video signals
spectral methods
abnormalities
Benchmarking
surveillance
Pixels
pixels
Trajectories
trajectories
evaluation

Cite this

Xu, Yuanping ; Lu, Li ; Xu, Zhijie ; He, Jia ; Wang, Jing ; Huang, Jian ; Lu, Jun. / Towards Intelligent Crowd Behavior Understanding through the STFD Descriptor Exploration. In: Sensing and Imaging. 2018 ; Vol. 19, No. 1. pp. 1-22.
@article{aa289533cf0b4d2685b189e4a52e8b91,
title = "Towards Intelligent Crowd Behavior Understanding through the STFD Descriptor Exploration",
abstract = "Realizing the automated and online detection of crowd anomalies from surveillance CCTVs is a research-intensive and application-demanding task. This research proposes a novel technique for detecting crowd abnormalities through analyzing the spatial and temporal features of input video signals. This integrated solution defines an image descriptor (named spatio-temporal feature descriptor—STFD) that reflects the global motion pattern of crowds over time. A designed convolutional neural network (CNN) has then been adopted to classify dominant or large-scale crowd abnormal behaviors. The work reported has focused on: (1) detecting moving objects in online (or near real-time) manner through spatio-temporal segmentations of crowds identified by the similarity of group trajectory structures in the temporal space and the foreground blocks based on the Gaussian mixture model in the spatial space; (2) dividing multiple clustered groups based on the spectral clustering methods through treating image pixels from segmented regions as dynamic particles; (3) creating STFD descriptor instances by calculating corresponding attributes such as collectiveness, stability, conflict and crowd density for individuals (particles) in the corresponding groups; (4) inputting generated STFD descriptor instances into the devised CNN to detect suspicious crowd behaviors. For the test and evaluation of the devised models and techniques, the PETS database has been selected as the primary experimental data sets. Results against benchmarking models and systems have shown promising advancements of this novel approach in terms of accuracy and efficiency for crowd anomaly detection.",
keywords = "Crowd abnormal behaviors, STFD descriptor, CNN, Group trajectory",
author = "Yuanping Xu and Li Lu and Zhijie Xu and Jia He and Jing Wang and Jian Huang and Jun Lu",
year = "2018",
month = "12",
doi = "10.1007/s11220-018-0201-3",
language = "English",
volume = "19",
pages = "1--22",
journal = "Sensing and Imaging",
issn = "1557-2064",
publisher = "Springer Science and Business Media Deutschland GmbH",
number = "1",

}

Towards Intelligent Crowd Behavior Understanding through the STFD Descriptor Exploration. / Xu, Yuanping; Lu, Li; Xu, Zhijie; He, Jia; Wang, Jing; Huang, Jian; Lu, Jun.

In: Sensing and Imaging, Vol. 19, No. 1, 17, 12.2018, p. 1-22.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Towards Intelligent Crowd Behavior Understanding through the STFD Descriptor Exploration

AU - Xu, Yuanping

AU - Lu, Li

AU - Xu, Zhijie

AU - He, Jia

AU - Wang, Jing

AU - Huang, Jian

AU - Lu, Jun

PY - 2018/12

Y1 - 2018/12

N2 - Realizing the automated and online detection of crowd anomalies from surveillance CCTVs is a research-intensive and application-demanding task. This research proposes a novel technique for detecting crowd abnormalities through analyzing the spatial and temporal features of input video signals. This integrated solution defines an image descriptor (named spatio-temporal feature descriptor—STFD) that reflects the global motion pattern of crowds over time. A designed convolutional neural network (CNN) has then been adopted to classify dominant or large-scale crowd abnormal behaviors. The work reported has focused on: (1) detecting moving objects in online (or near real-time) manner through spatio-temporal segmentations of crowds identified by the similarity of group trajectory structures in the temporal space and the foreground blocks based on the Gaussian mixture model in the spatial space; (2) dividing multiple clustered groups based on the spectral clustering methods through treating image pixels from segmented regions as dynamic particles; (3) creating STFD descriptor instances by calculating corresponding attributes such as collectiveness, stability, conflict and crowd density for individuals (particles) in the corresponding groups; (4) inputting generated STFD descriptor instances into the devised CNN to detect suspicious crowd behaviors. For the test and evaluation of the devised models and techniques, the PETS database has been selected as the primary experimental data sets. Results against benchmarking models and systems have shown promising advancements of this novel approach in terms of accuracy and efficiency for crowd anomaly detection.

AB - Realizing the automated and online detection of crowd anomalies from surveillance CCTVs is a research-intensive and application-demanding task. This research proposes a novel technique for detecting crowd abnormalities through analyzing the spatial and temporal features of input video signals. This integrated solution defines an image descriptor (named spatio-temporal feature descriptor—STFD) that reflects the global motion pattern of crowds over time. A designed convolutional neural network (CNN) has then been adopted to classify dominant or large-scale crowd abnormal behaviors. The work reported has focused on: (1) detecting moving objects in online (or near real-time) manner through spatio-temporal segmentations of crowds identified by the similarity of group trajectory structures in the temporal space and the foreground blocks based on the Gaussian mixture model in the spatial space; (2) dividing multiple clustered groups based on the spectral clustering methods through treating image pixels from segmented regions as dynamic particles; (3) creating STFD descriptor instances by calculating corresponding attributes such as collectiveness, stability, conflict and crowd density for individuals (particles) in the corresponding groups; (4) inputting generated STFD descriptor instances into the devised CNN to detect suspicious crowd behaviors. For the test and evaluation of the devised models and techniques, the PETS database has been selected as the primary experimental data sets. Results against benchmarking models and systems have shown promising advancements of this novel approach in terms of accuracy and efficiency for crowd anomaly detection.

KW - Crowd abnormal behaviors

KW - STFD descriptor

KW - CNN

KW - Group trajectory

UR - http://www.scopus.com/inward/record.url?scp=85046673888&partnerID=8YFLogxK

U2 - 10.1007/s11220-018-0201-3

DO - 10.1007/s11220-018-0201-3

M3 - Article

VL - 19

SP - 1

EP - 22

JO - Sensing and Imaging

JF - Sensing and Imaging

SN - 1557-2064

IS - 1

M1 - 17

ER -