An encoder–decoder network for land cover classification using a fusion of aerial images and photogrammetric point clouds

Abstract

Land cover information is becoming more important in urban planning, change detection, and management. The fusion of point clouds and images increases the accuracy of land use classification by utilising the advantages of both modalities. Similar structures such as buildings and roads, low and high vegetation, and impervious and bare regions are not too much discriminative. Models fail to discriminate these classes leading to misclassifications, false detections, and unreliable land cover maps. Therefore, this research proposes the fusion of dense point clouds and multi-spectral images based on a dual-stream deep convolutional model by adding vegetation and elevation information to spectral information. To fuse both modalities' features, a dual-stream deep neural network based on Deeplabv3+ architecture is implemented. In addition, the Xception (Extreme Inception) model is considered as a backbone and feature extractor. The model performance is evaluated with F1-Score and Overall Accuracy. 93.4% Overall Accuracy and F1-Score are achieved after adding height and vegetation information to the model. Results indicate improvements in all indexes, meaning that data fusion with the proposed model outperforms the existing state-of-the-art models.

Keywords:

• Urban planning
• Land cover classification
• DeepLabv3+
• Xception
• Image-LiDAR fusion
• Deep learning
• Dual-stream deep neural network

Disclosure statement

No potential conflict of interest was reported by the author(s).

Correction Statement

This article has been corrected with minor changes. These changes do not impact the academic content of the article.

Additional information

Notes on contributors

Soheil Majidi

Soheil Majidi is a graduate student of photogrammetry and remote sensing engineering from the University of Tehran. He received his BSc in the field of surveying and geomatics engineering at the Geomatics Engineering Department, Faculty of Engineering, University of Zanjan, in 2020. Currently, he is working on 3D deep learning and his research interests are non-rigid and rigid point cloud registration, deformation analysis, and point cloud segmentation using deep learning.

Ghazale Babapour

Ghazale Babapour is a master student in Remote Sensing at the School of Surveying and Geospatial Engineering, University of Tehran. She completed her B.S. in Geomatics Engineering at the Geomatics Engineering Department, Faculty of Engineering, University of Zanjan, in 2021. Her research interests include UAV and satellite image classification, machine/deep learning, and especially in her master's thesis, monitoring, analyzing and prediction of land subsidence.

Reza Shah-Hosseini

Reza Shah-Hosseini is an associate professor at the School of Surveying and Geospatial Engineering, University of Tehran. He received his BS degree in surveying and geomatics engineering, and his MS and PhD degrees in remote sensing from the School of Surveying and Geospatial Engineering, College of Engineering, University of Tehran, in 2007, 2010, and 2016, respectively. His current research interests are in the area of remote sensing applications, image classification, change detection, agricultural remote sensing, natural hazard assessment, pattern recognition, machine learning, and deep learning.