Road and Vegetation Extraction from Aerial Images through Semantic Segmentation

Abstract

Images captured by satellites and unmanned aerial vehicles (UAVs) have the unique ability to present the view from an aerial perspective and see the earth from above. These images have a great significance that can be noticed from their wide range of applications in various sectors. Vision-based solutions have gained popularity with the help of machine learning algorithms. Object segmentation from aerial images plays a vital role in urban design, georeferencing, automated vehicle navigation, geospatial data integration, intelligent transportation system, and disaster management. Therefore, objects such as road and vegetation extractions are worth studying and have recently been of growing interest among researchers. Moreover, on account of various research challenges, such as diversity of object appearance, light illumination variation (due to timing of the day and weather) causing vision issues, scaling effect (due to variation in the height of flight of UAV), and occlusion (due to several objects), automatic road and vegetation extraction from remote sensing (RS) images is still at its infancy. Similarly, performing these tasks on board under limited resources to achieve real-time data processing is quite challenging. The existing deep learning frameworks also need massive computational and storage power to complete the segmentation tasks. Hence, in this doctoral research, an attempt has been made to analyze the technical challenges in the road and vegetation segmentation task concerning space-borne and air-borne platforms while providing computer vision-based deep learning solutions to overcome the current issues for semantic-level optimization. This research primarily involves the proposition of lightweight deep semantic segmentation models to perform pixel-level classification over satellites and UAVs on-board. The dissertation proposes four newly developed deep learning methodologies for RS image segmentation tasks, especially to segment the road and vegetation regions, including one proposed UAV-based RS image dataset. The dissertation starts by providing the solution for the classical (satellite-inspired) remote-sensing problems for road extraction. The first contribution introduces a superpixel-aided CNN architecture that can be deployed over the proposed three-layer IoT framework for achieving real-time road segmentation using satellite RS images. The experimentally obtained results show that the ensembling architecture not only performs better but also shows its efficiency with optimized space complexity, which is essential for IoT-based network frameworks. However, the satellite images may not be suitable for various RS applications, such as vegetation disease detection, a class of plant detection, visual surveillance in forests, etc. This can be analyzed from the operational height of the satellites that produce poor-resolution remote sensing images, which will be unsuitable for some RS tasks demanding higher-resolution imageries. Moreover, the cloud may also become an obstacle, especially in tropical regions. UAV-based aerial images have become a savior for such situations. The scarcity of UAV-based semantic segmentation datasets becomes one of the motivations to prepare and benchmark a UAV segmentation dataset known as the “NITRDrone dataset,” which contributes to the second contribution. While benchmarking the dataset, it has been observed that the existing encoder-decoder CNN architectures suffer from three significant issues: degradation problem, vanishing gradient problem, and overfitting. Therefore, as an immediate improvement, the third contribution of the thesis proposes a skip connection-based encoder-decoder architecture, also known as AerialSegNet. The proposed AerialSegNet significantly reduces the number of trainable parameters while improving the accuracy in terms of IoU and F-Score compared to state-of-the-art methods. However, AerialSegNet fails to capture scale-invariant features leading to poor generalization for several small-scale objects on the ground. To overcome these related issues, the fourth contribution involves the development of a superpixel-aided multiscale CNN architecture that improves the classification accuracy to a great extent, however, with an increase in the number of trainable parameters. Therefore, in the final and fifth contribution, a lightweight deep CNN architecture known as LW-AerialSegNet is suggested. The proposed lightweight architecture can produce better prediction accuracy while having 70% fewer parameters than the proposed AerialSegNet (in the third contribution) and state-of-the-art methods, solving all three issues of deep semantic segmentation architectures simultaneously. The performance of all the proposed methodologies is validated experimentally with multiple publicly available datasets, and the results are compared both quantitatively and quantitatively with state-of-the-art methods. The research outcomes can be extended further to perform remote sensing applications, such as urban planning, precision agriculture, forest management, disaster management, etc.