Fusion and Super-Resolution of Remote Sensing Images Using Deep Learning and Gram-Schmidt Orthogonalization
DOI:
https://doi.org/10.52562/injoes.2026.1632Keywords:
Sentinel-2, pan-sharpening, convolutional neural network, aerial imagery, urban vegetationAbstract
Multispectral satellite images contain rich spectral information but often have low spatial resolution. Super-resolution of these images is important for resource development, environmental protection, disaster prevention, regional ecology, and change detection. Deep learning models, including convolutional neural networks (CNNs), have been widely applied to the super-resolution of satellite images, including Sentinel-2 images. However, their output cannot exceed the maximum spatial resolution of the sensor. Here, we propose a method that first enhances the 20 m Sentinel-2 bands to 10 m using a CNN and then fuses them with 1 m aerial images by Gram-Schmidt orthogonal transformation and its inverse, while preserving multispectral information. The method was tested on the fusion of aerial and Sentinel-2 images to visualize vegetation distribution in urban areas. The proposed network achieved a root mean square error (RMSE) of 43.5 and a correlation coefficient (CC) of 0.941, compared with 123.4 and 0.811 for bicubic interpolation and 117.8 and 0.866 for area-to-point regression kriging (ATPRK). The fused images appeared as sharp as the aerial images while preserving the shape of the original spectral curves.
Downloads
References
Aiazzi, B., Baronti, S., & Selva, M. (2007). Improving component substitution pansharpening through multivariate regression of MS + Pan data. IEEE Transactions on Geoscience and Remote Sensing, 45(10), 3230-3239. https://doi.org/10.1109/TGRS.2007.901007
Björck, Å. (1967). Solving linear least squares problems by Gram-Schmidt orthogonalization. BIT, 7(1), 1–21. https://doi.org/10.1007/BF01934122
Cai, D., & Zhang, P. (2022). T3SR: Texture transfer transformer for remote sensing image superresolution. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 15, 7346–7358. https://doi.org/10.1109/jstars.2022.3198557
Dong, X., Xi, Z., Sun, X., & Gao, L. (2019). Transferred multi-perception attention networks for remote sensing image super-resolution. Remote Sensing, 11(23), Article 2857. https://doi.org/10.3390/rs11232857
Drusch, M., Del Bello, U., Carlier, S., Colin, O., Fernandez, V., Gascon, F., ... & Bargellini, P. (2012). Sentinel-2: ESA's optical high-resolution mission for GMES operational services. Remote sensing of Environment, 120, 25-36. https://doi.org/10.1016/j.rse.2011.11.026
Gargiulo, M., Mazza, A., Gaetano, R., Ruello, G., & Scarpa, G. (2019). Fast super-resolution of 20 m Sentinel-2 bands using convolutional neural networks. Remote Sensing, 11(22), Article 2635. https://doi.org/10.3390/rs11222635
Kingma, D. P., & Ba, J. (2014). Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980. https://doi.org/10.48550/arXiv.1412.6980
Lanaras, C., Bioucas-Dias, J., Galliani, S., Baltsavias, E., & Schindler, K. (2018). Super-resolution of Sentinel-2 images: Learning a globally applicable deep neural network. ISPRS Journal of Photogrammetry and Remote Sensing, 146, 305–319. https://doi.org/10.1016/j.isprsjprs.2018.09.018
Lei, S., Shi, Z., & Mo, W. (2022). Transformer-based multistage enhancement for remote sensing image super-resolution. IEEE Transactions on Geoscience and Remote Sensing, 60, Article 5615611. https://doi.org/10.1109/tgrs.2021.3136190
Lu, T., Wang, J., Zhang, Y., Wang, Z., & Jiang, J. (2019). Satellite image super-resolution via multi-scale residual deep neural network. Remote Sensing, 11(13), Article 1588. https://doi.org/10.3390/rs11131588
Ma, W., Pan, Z., Yuan, F., & Lei, B. (2019). Super-resolution of remote sensing images via a dense residual generative adversarial network. Remote Sensing, 11(21), Article 2578. https://doi.org/10.3390/rs11212578
Romero, L. S., Marcello, J., & Vilaplana, V. (2020). Super-resolution of Sentinel-2 imagery using generative adversarial networks. Remote Sensing, 12(15), Article 2424. https://doi.org/10.3390/rs12152424
Vivone, G., Alparone, L., Chanussot, J., Dalla Mura, M., Garzelli, A., Licciardi, G. A., ... & Wald, L. (2014). A critical comparison among pansharpening algorithms. IEEE Transactions on Geoscience and Remote Sensing, 53(5), 2565-2586. https://doi.org/10.1109/TGRS.2014.2361734
Wang, Y., Shao, Z., Lu, T., Huang, X., Wang, J., Chen, X., Huang, H., & Zuo, X. (2023). Remote sensing image super-resolution via multi-scale texture transfer network. Remote Sensing, 15(23), Article 5503. https://doi.org/10.3390/rs15235503
Wang, Q., Shi, W., Li, Z., & Atkinson, P. M. (2016). Fusion of Sentinel-2 images. Remote sensing of environment, 187, 241-252. https://doi.org/10.1016/j.rse.2016.10.030
Xiong, Y., Guo, S., Chen, J., Deng, X., Sun, L., Zheng, X., & Xu, W. (2020). Improved SRGAN for remote sensing image super-resolution across locations and sensors. Remote Sensing, 12(8), Article 1263. https://doi.org/10.3390/rs12081263
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Sin-Gil Kim, Il-Kwang Jo, Ji-Chon Ro
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
