A Modified Activation Function for Deep Convolutional Neural Network and Its Application to Condition Monitoring

Convolutional Neural Network (CNN) is a deep learning model which has been an active research topic and applied extensively to vibration data for condition monitoring (CM). In CNN, hyper-parameters, such as activation function, have a significant effect on the training task and, consequently, on the overall performance of the network. The existing activation functions have some limitations, such as vanishing gradient problem, dead neurons, and fixed gradient value. In order to address the reported issues, this paper proposes an improved activation function for deep CNN, namely (IReLU-Tanh). It adopts the advantage of ReLU function in covering the positive region, also by taking the properties of the negative region from the Tanh function. Therefore, the proposed IReLU-Tanh function addresses the existing shortcomings, both vanishing gradient, dead neurons, and fixed gradient value. To prove its effectiveness, the proposed IReLU-Tanh function is evaluated based on both simulated and experimental vibration data. Results show that the proposed IReLU-Tanh function enhances remarkably the overall performance of the network in two aspects; firstly, in training task, the model parameters can reach the optimum values with lower learning errors compared to other functions, so the network can learn effectively the hidden features. Secondly, it improves the overall accuracy of the classification task and yields robust detection and diagnosis performance when compared against the other activation functions including Tanh, ReLU, LReLU, and ELU.