A Modified Activation Function for Deep Convolutional Neural Network and Its Application to Condition Monitoring

Ibrahim Alqatawneh, Khalid Rabeyee, Chao Zhang, Guojin Feng, Fengshou Gu, Andrew D. Ball

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

Convolutional Neural Network (CNN) is a deep learning model which has been an active research topic and applied extensively to vibration data for condition monitoring (CM). In CNN, hyper-parameters, such as activation function, have a significant effect on the training task and, consequently, on the overall performance of the network. The existing activation functions have some limitations, such as vanishing gradient problem, dead neurons, and fixed gradient value. In order to address the reported issues, this paper proposes an improved activation function for deep CNN, namely (IReLU-Tanh). It adopts the advantage of ReLU function in covering the positive region, also by taking the properties of the negative region from the Tanh function. Therefore, the proposed IReLU-Tanh function addresses the existing shortcomings, both vanishing gradient, dead neurons, and fixed gradient value. To prove its effectiveness, the proposed IReLU-Tanh function is evaluated based on both simulated and experimental vibration data. Results show that the proposed IReLU-Tanh function enhances remarkably the overall performance of the network in two aspects; firstly, in training task, the model parameters can reach the optimum values with lower learning errors compared to other functions, so the network can learn effectively the hidden features. Secondly, it improves the overall accuracy of the classification task and yields robust detection and diagnosis performance when compared against the other activation functions including Tanh, ReLU, LReLU, and ELU.

Original languageEnglish
Title of host publicationProceedings of IncoME-V & CEPE Net-2020
Subtitle of host publicationCondition Monitoring, Plant Maintenance and Reliability
EditorsDong Zhen, Dong Wang, Tianyang Wang, Hongjun Wang, Baoshan Huang, Jyoti K. Sinha, Andrew David Ball
Place of PublicationCham
PublisherSpringer Nature Switzerland AG
Pages895-909
Number of pages15
Volume105
Edition1st
ISBN (Electronic)9783030757939
ISBN (Print)9783030757922
DOIs
Publication statusPublished - 16 May 2021
Event5th International Conference on Maintenance Engineering and the 2020 Annual Conference of the Centre for Efficiency and Performance Engineering Network - Zhuhai, China
Duration: 23 Oct 202025 Oct 2020
Conference number: 5
https://link.springer.com/book/10.1007/978-3-030-75793-9#about

Publication series

NameMechanisms and Machine Science
Volume105
ISSN (Print)2211-0984
ISSN (Electronic)2211-0992

Conference

Conference5th International Conference on Maintenance Engineering and the 2020 Annual Conference of the Centre for Efficiency and Performance Engineering Network
Abbreviated titleIncoME-V and CEPE Net-2020
CountryChina
CityZhuhai
Period23/10/2025/10/20
Internet address

Fingerprint Dive into the research topics of 'A Modified Activation Function for Deep Convolutional Neural Network and Its Application to Condition Monitoring'. Together they form a unique fingerprint.

Cite this