Multiscale Spectral–Spatial Capsule Neural Network for Hyperspectral Image Classification

Deep learning models have recently shown good performance in the hyperspectral remote sensing image classification tasks. In particular, a capsule network (CapsNet) was introduced as a powerful alternative to convolutional neural networks (CNNs). The CapsNet adopts vector neuron and encode the spatial relationship of features in an image, which exhibits encouraging performance. Motivated by CapsNet, this paper presents a novel capsule network based on multiscale spectral–spatial features to improve the performance for hyperspectral images (HSIs) classification. First, multi-scale features are extracted from the hyperspectral data in 1D spectral, 2D spatial and 3D spatial-spectral cubes, and their primary capsule neurons are constructed separately. Then, a multi-head attention mechanism is introduced to model the association of all primary capsule neurons from multiple dimensions to efficiently extract finer-grained multi-scale spatial-spectral information. Finally, the multiscale capsule neurons are updated using a dynamic routing agreement to obtain more discriminative high-level capsule features thus improving the classification accuracy. Experimental results on two commonly used HSIs datasets (Indian Pines, Pavia University) demonstrate that the proposed model can achieve better performance compared with other state-of-the art deep-learning-based approaches.