Autoencoder for anomaly detection in hperspectral imagery

Leveraging spectral and spatial information to improve autoencoders, enhancing the identification of abnormal pixels

Hyperspectral imaging (HSI) from spaceborne, airborne, and drone platforms has emerged as a transformative Earth observation technology and provides spatially distributed detailed spectral information across potentially hundreds of contiguous wavelength bands. By capturing the unique spectral signatures of materials, hyperspectral sensors enable the precise identification and characterisation of vegetation, soils, minerals, water quality, and man-made objects. Satellite-based systems offer large-scale and repetitive monitoring for global environmental and climate studies, while airborne sensors provide higher spatial resolution and flexibility for regional applications. Drone-mounted HSI cameras further enhance observation capabilities by delivering ultra-high-resolution data tailored to local-scale investigations. Together, these platforms support a wide range of applications, including precision agriculture, ecosystem monitoring, biodiversity assessment, mineral exploration, environmental pollution detection, disaster management, urban mapping, and infrastructure inspection. The synergy between space, air, and drone observations enables multi-scale analyses, fostering improved understanding and sustainable management of natural and built environments.

A hyperspectral image is arranged by a three-dimensional (3D) data cube (Fig. 1) acquired by using imaging and spectroscopy techniques.^2,3 HSI provides 2D spatial information along with comprehensive spectral information of objects mapped to the third dimension. Thus, spectral-spatial information can explore the fine-detailed properties of objects in the images. The purpose of anomaly detection is to find abnormal samples whose distributions are inconsistent with most instances in the dataset.⁶ Hyperspectral anomaly detection (HAD), which is used to identify spectrally distinct (abnormal) pixels, is crucial in many applications, such as environmental monitoring, precision agriculture, and surveillance. In case of environmental monitoring, HAD enables early detection of forest stress, pollution, and oil spills. In precision agriculture, they facilitate the early detection of crop diseases, nutrient deficiencies, and water stress, and in surveillance, they help to identify camouflaged vehicles and concealed installations. The combination of spatial and spectral information gives complementary benefits for better HAD. Many methods have been developed for HAD; these can be classified into probability-distribution-based, representation-based, and deep learning-based ones.⁵

Enhancing autoencoders for HAD

Deep learning methods can be grouped into supervised learning and unsupervised learning frameworks. Supervised learning methods are distinguished by their reliance on labelled training data for model development. In contrast, unsupervised learning methods do not need to know anything about the target pixels a priori. A well-known unsupervised feature extraction model is autoencoder (AE), and it has been widely applied for HAD.^2,6 The idea of AE is to consider that most of the pixels in HSI are background, while the anomalous pixels are only a small part of the image. Reasonably, when a hyperspectral image is used in the AE model as input, the model generally learns the background. Then AE simultaneously reconstructs both the anomaly targets and the background, but the lack of prior information limits the ability to detect anomalies⁶ and can provide many false positive responses.

Recently, we developed an improved version of AE¹ that combines a 3D Convolutional Autoencoder and Spectral-Spatial Attention (SSA). The new approach employs 3D convolutional layers to create a compact latent space from the input image, while the decoder uses deconvolution layers to recover the input image (Fig. 2). The 3D convolutional layers convolve across spectral and spatial dimensions, thus leveraging both the spectral and spatial hierarchy of image features. 3D convolutional AE preserves spectral-spatial patterns well but focuses mainly on local neighbourhoods. It treats all spectral channels (bands) equally over the whole image but with no extra ‘attention’ toward potentially important ones. Therefore, they may fail to focus on informative bands. To focus on important bands, we attach multi-head spectral attention and multi-scale spatial attention, where spectral attention dynamically reweights each spectral band. Hence, the network emphasises discriminative wavelengths and suppresses noise. Besides, multi-scale spatial attention focuses on where (which pixels or regions) to emphasise or suppress, which enhances local contrast and spatial coherence. Details of the developed algorithm can be found in.¹

Demonstrating performance using the ABU Urban-II dataset

To show the performance of the developed AE algorithm, we build an error map based on the well-known robust Mahalanobis-statistical distance metric. We experimented on several benchmark HSI datasets. Here is an example to show the performance of the new method. The dataset that we used was acquired using the airborne visible/infrared imaging spectrometer (AVIRIS) over the Texas Coast area in the USA. The dataset is known as the ABU Urban-II dataset (Texas Coast-II or ABU-U2⁴). It has 207 spectral bands within 400 to 2500 nm wavelengths, and the spatial resolution is 17.2 m/pixel. The size of the dataset is 100*100 pixels. The buildings are treated as anomalies and comprise only 155 pixels. The false-colour image and the ground truth (GT) of this dataset are shown in Fig. 3(a) and 3(b), respectively. We calculate the area under the curve (AUC), which is one of the most widely used metrics to evaluate the performance of HAD algorithms. The new algorithm achieves an AUC of 99.91%. Fig.3(c) shows that the new algorithm effectively suppresses noise, highlights semantically meaningful features, and isolates anomalies.

A particular focus for our upcoming HAD work will be in precision agriculture, e.g., the detection of weeds in crop fields, or in forestry, e.g., the detection of healthy and stressed trees in groups of trees belonging to a single species.

References

1. Akter, S., Teferle, F., Nurunnabi, A., 3D Convolutional autoencoder with spectral-spatial attention (3D CAE-SSA) for anomaly detection in hyperspectral images, 14th EARSeL Workshop on Imaging Spectroscopy, 2 – 4 June 2026, Helsinki, Finland.
2. Guo Q., Yi C, Zhang L., Zhang Y., Hu S., Liu X., Robust multi-stage progressive autoencoder for hyperspectral anomaly detection, Int. J. Appl. Earth Obs. Geoinf., vol. 134 (104200), 2024.
3. Qi, W., Huang, C., Wang, Y., Zhang, X., Sun, W., Zhang, L., Global-local three- dimensional convolutional transformer network for hyperspectral image classification. IEEE Trans. Geosci. Remote Sens, vol. 61, 2023.
4. Qin, H., Shen, Q., Zeng, H., Chen, Y., Lu, G., Generalized nonconvex low-rank tensor representation for hyperspectral anomaly detection. IEEE Trans. Geosci. Remote Sens., vol. 61, 2023.
5. Su H., Wu Z., Zhang H., Du Q., Hyperspectral anomaly detection: A survey, EEE Geosci. Remote Sens. Mag., vol. 10(1), pp. 64–90, 2022.
6. Xie W., Liu B., Li Y., Lei J., Du Q., Autoencoder and adversarial learning based semi supervised background estimation for hyperspectral anomaly detection, IEEE Trans. Geosci. Remote Sens., vol. 58(8), pp. 5416–5427, 2020.
7. Shaw, G. A. and Burke, H. K., Spectral imaging for remote sensing. Lincoln laboratory journal, 14(1), pp. 3-28, 2003

Acknowledgments

Salena Akter is fully funded by the University of Luxembourg. Project PRECISION is funded by the Luxembourg National Research Fund (FNR) and the Ministry of Agriculture, Food and Viticulture (MAAV) under the Agriculture BRIDGES Call 2022, reference 17949865.

This article will feature in our upcoming Space Special Focus Publication.