[go: up one dir, main page]

 
 
remotesensing-logo

Journal Browser

Journal Browser

Computer Vision-Based Methods and Tools in Remote Sensing

A special issue of Remote Sensing (ISSN 2072-4292). This special issue belongs to the section "Environmental Remote Sensing".

Deadline for manuscript submissions: 31 December 2024 | Viewed by 18716

Special Issue Editor


E-Mail Website
Guest Editor
Department of Computer Science and Biomedical Informatics, School of Science, Campus of Lamia, University of Thessaly, GR-35131 Lamia, Greece
Interests: pattern recognition; computer vision; expert systems
Special Issues, Collections and Topics in MDPI journals

Special Issue Information

Dear Colleagues,

The progress of remote sensing imaging has been closely associated with computer vision and pattern recognition. Land cover mapping, target detection, change detection and boundary extraction, as well as pattern inference from time-series of imaging data, pose challenges for traditional image computer vision and pattern recognition tasks, such as image clustering, classification and segmentation. Engineered feature vectors have been applied for the analysis of optical, SAR, multispectral or hyperspectral images, as well as point clouds. Later, visual dictionaries, the so-called bag-of-visual-words models (BoVW), incorporated the statistics associated with each problem at hand. In the context of semantic segmentation, numerous approaches, including active contours, Markov random fields (MRF) and superpixels, have been combined with descriptors or BoVWs and applied on remote sensing data. The rapid evolution of powerful GPUs and the availability of large datasets aided extraordinary advances in deep-learning-based computer vision, starting from the performance breakthrough of AlexNet in ILSVRC 2012. This new paradigm reinvigorated the interest in computational tools for remote sensing. Numerous works are regularly published for the analysis of remote sensing data with deep neural networks. Convolutional neural networks (CNNs) and derivative architectures, such as VGG, ResNet and Inception, play a prominent role in this direction. Another deep learning branch in remote sensing consists of applications of recurrent neural networks (RNNs), including long short-term memory (LSTM) networks, on time-series of images.

This Special Issue aims to explore state-of-the-art computer vision and pattern recognition applications in remote sensing. Research contributions, including surveys, are welcome. In particular, novel contributions that cover, but are not limited to, the following application domains are welcome:

  • Land cover mapping;
  • Target detection;
  • Change detection;
  • Boundary extraction;
  • Pattern analysis on time-series of imaging data;
  • Works carried out at all scales and in all environments, including surveys and comparative studies, as well as the description of new methodologies, best practices, advantages and limitations for computational tools in remote sensing.

Prof. Dr. Michalis Savelonas
Guest Editor

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Remote Sensing is an international peer-reviewed open access semimonthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 2700 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

Keywords

  • land cover mapping
  • target detection
  • change detection
  • point-clouds
  • LiDAR
  • SAR imaging
  • multispectral imaging
  • hyperspectral imaging
  • computer vision
  • pattern recognition
  • deep learning
  • machine learning
  • image analysis
  • 3D shape analysis
  • time-series analysis

Benefits of Publishing in a Special Issue

  • Ease of navigation: Grouping papers by topic helps scholars navigate broad scope journals more efficiently.
  • Greater discoverability: Special Issues support the reach and impact of scientific research. Articles in Special Issues are more discoverable and cited more frequently.
  • Expansion of research network: Special Issues facilitate connections among authors, fostering scientific collaborations.
  • External promotion: Articles in Special Issues are often promoted through the journal's social media, increasing their visibility.
  • e-Book format: Special Issues with more than 10 articles can be published as dedicated e-books, ensuring wide and rapid dissemination.

Further information on MDPI's Special Issue polices can be found here.

Published Papers (7 papers)

Order results
Result details
Select all
Export citation of selected articles as:

Research

30 pages, 25238 KiB  
Article
Unsupervised Color-Based Flood Segmentation in UAV Imagery
by Georgios Simantiris and Costas Panagiotakis
Remote Sens. 2024, 16(12), 2126; https://doi.org/10.3390/rs16122126 - 12 Jun 2024
Viewed by 707
Abstract
We propose a novel unsupervised semantic segmentation method for fast and accurate flood area detection utilizing color images acquired from unmanned aerial vehicles (UAVs). To the best of our knowledge, this is the first fully unsupervised method for flood area segmentation in color [...] Read more.
We propose a novel unsupervised semantic segmentation method for fast and accurate flood area detection utilizing color images acquired from unmanned aerial vehicles (UAVs). To the best of our knowledge, this is the first fully unsupervised method for flood area segmentation in color images captured by UAVs, without the need of pre-disaster images. The proposed framework addresses the problem of flood segmentation based on parameter-free calculated masks and unsupervised image analysis techniques. First, a fully unsupervised algorithm gradually excludes areas classified as non-flood, utilizing calculated masks over each component of the LAB colorspace, as well as using an RGB vegetation index and the detected edges of the original image. Unsupervised image analysis techniques, such as distance transform, are then applied, producing a probability map for the location of flooded areas. Finally, flood detection is obtained by applying hysteresis thresholding segmentation. The proposed method is tested and compared with variations and other supervised methods in two public datasets, consisting of 953 color images in total, yielding high-performance results, with 87.4% and 80.9% overall accuracy and F1-score, respectively. The results and computational efficiency of the proposed method show that it is suitable for onboard data execution and decision-making during UAV flights. Full article
(This article belongs to the Special Issue Computer Vision-Based Methods and Tools in Remote Sensing)
Show Figures

Graphical abstract

Graphical abstract
Full article ">Figure 1
<p>Sample images from the Flood Area dataset (<b>top</b>) and their corresponding ground truths (<b>bottom</b>).</p>
Full article ">Figure 2
<p>Sample images from the Flood Semantic Segmentation dataset (<b>top</b>) and their corresponding ground truths (<b>bottom</b>).</p>
Full article ">Figure 3
<p>Overview of the proposed approach.</p>
Full article ">Figure 4
<p>The original images (from the Flood Area dataset) and the corresponding RGBVI masks on their right side. The masks show detected greenery with dim gray color. Examples are presented for (<b>a</b>) urban areas, (<b>b</b>) rural areas, and (<b>c</b>) poor or failed greenery detection. The remaining potential flood areas are shown in cyan.</p>
Full article ">Figure 5
<p>Blue and red curves correspond on the average value of (<b>a</b>) L, (<b>b</b>) A, and (<b>c</b>) B color components computed on flood and background pixels, respectively, for each image of the Flood Area dataset, sorted in ascending order. The yellow curves show the corresponding <math display="inline"><semantics> <msub> <mo>Δ</mo> <mi>C</mi> </msub> </semantics></math> threshold.</p>
Full article ">Figure 6
<p>Original images from the Flood Area dataset and their corresponding LAB components masks <math display="inline"><semantics> <mrow> <msub> <mi>M</mi> <mi>L</mi> </msub> <mo>,</mo> <msub> <mi>M</mi> <mi>A</mi> </msub> <mo>,</mo> <msub> <mi>M</mi> <mi>B</mi> </msub> </mrow> </semantics></math>, and <math display="inline"><semantics> <msub> <mi>M</mi> <mrow> <mi>E</mi> <mi>d</mi> <mi>g</mi> <mi>e</mi> </mrow> </msub> </semantics></math> from left to right. Note that the edges in <math display="inline"><semantics> <msub> <mi>M</mi> <mrow> <mi>E</mi> <mi>d</mi> <mi>g</mi> <mi>e</mi> </mrow> </msub> </semantics></math> are dilated for illustration purposes. Non-flood areas are depicted with dim gray color, whereas remaining potential flood areas are shown in cyan.</p>
Full article ">Figure 7
<p>Probability maps (column 4) obtained using potential flood areas of <math display="inline"><semantics> <msub> <mi>M</mi> <mrow> <mi>F</mi> <mi>i</mi> <mi>n</mi> <mi>a</mi> <mi>l</mi> </mrow> </msub> </semantics></math> (column 2), weight maps (column 3), as generated by the distance transform and the corresponding images from the Flood Area dataset (column 1). Potential flood area is shown in cyan, and non-flood area in dim gray color. The weights and probabilities range from 0 (dark blue color) to 1 (red color).</p>
Full article ">Figure 8
<p>(<b>a</b>) Original image from the Flood Area dataset, (<b>b</b>) the applied hysteresis thresholding on the decisive probability map of the potential flood area, and (<b>c</b>) the final segmentation mask. (<b>b</b>) In red and blue are the pixels with <math display="inline"><semantics> <mrow> <mi>P</mi> <mi>M</mi> </mrow> </semantics></math>(<span class="html-italic">p</span>) &gt; <math display="inline"><semantics> <msub> <mi>T</mi> <mi>H</mi> </msub> </semantics></math> and <math display="inline"><semantics> <msub> <mi>T</mi> <mi>L</mi> </msub> </semantics></math> &lt; <math display="inline"><semantics> <mrow> <mi>P</mi> <mi>M</mi> </mrow> </semantics></math>(<span class="html-italic">p</span>) ≤ <math display="inline"><semantics> <msub> <mi>T</mi> <mi>H</mi> </msub> </semantics></math>, respectively. Cyan-colored pixels are with <math display="inline"><semantics> <mrow> <mi>P</mi> <mi>M</mi> </mrow> </semantics></math>(<span class="html-italic">p</span>) ≤ <math display="inline"><semantics> <msub> <mi>T</mi> <mi>L</mi> </msub> </semantics></math>, they do not surpass the lower threshold, and are subsequently classified as background. The non-flood areas, according to the <math display="inline"><semantics> <msub> <mi>M</mi> <mrow> <mi>F</mi> <mi>i</mi> <mi>n</mi> <mi>a</mi> <mi>l</mi> </mrow> </msub> </semantics></math> mask, are colored with dim gray pixels. (<b>c</b>) The last column shows the final segmentation obtained from our proposed method, where the flood is in blue and the background is in dim gray color.</p>
Full article ">Figure 9
<p>High-performance results of the proposed flood segmentation method from the Flood Area dataset. Original images, ground truth, and the final segmentation of our proposed method (UFS-HT-REM).</p>
Full article ">Figure 10
<p>Satisfactory results of the proposed flood segmentation method from the Flood Area dataset. Original images, ground truth, and our proposed method’s (UFS-HT-REM) final segmentation.</p>
Full article ">Figure 11
<p>Poor segmentations resulting from the proposed methodology (UFS-HT-REM) from the Flood Area dataset. Original images, ground truth, and the final segmentation of our proposed method.</p>
Full article ">Figure 12
<p>The average values of <math display="inline"><semantics> <mrow> <mi>A</mi> <mi>C</mi> <mi>C</mi> </mrow> </semantics></math>, <math display="inline"><semantics> <mrow> <mi>R</mi> <mi>E</mi> <mi>C</mi> </mrow> </semantics></math>, <math display="inline"><semantics> <mrow> <mi>P</mi> <mi>R</mi> </mrow> </semantics></math>, <math display="inline"><semantics> <msub> <mi>F</mi> <mn>1</mn> </msub> </semantics></math>, and <math display="inline"><semantics> <mover> <msub> <mi>F</mi> <mn>1</mn> </msub> <mo>¯</mo> </mover> </semantics></math> computed on the Flood Area dataset for different values of (<b>a</b>) <math display="inline"><semantics> <msub> <mi>T</mi> <mi>L</mi> </msub> </semantics></math> (with <math display="inline"><semantics> <msub> <mi>T</mi> <mi>H</mi> </msub> </semantics></math> = 0.75) and (<b>b</b>) <math display="inline"><semantics> <msub> <mi>T</mi> <mi>H</mi> </msub> </semantics></math> (with <math display="inline"><semantics> <msub> <mi>T</mi> <mi>L</mi> </msub> </semantics></math> = 0.01).</p>
Full article ">Figure 13
<p>Representative results of the proposed methodology (UFS-HT-REM) from the Flood Semantic Segmentation dataset. Original images, ground truth, and the final segmentation of the proposed method are shown from left to right.</p>
Full article ">
20 pages, 6258 KiB  
Article
Locating and Grading of Lidar-Observed Aircraft Wake Vortex Based on Convolutional Neural Networks
by Xinyu Zhang, Hongwei Zhang, Qichao Wang, Xiaoying Liu, Shouxin Liu, Rongchuan Zhang, Rongzhong Li and Songhua Wu
Remote Sens. 2024, 16(8), 1463; https://doi.org/10.3390/rs16081463 - 20 Apr 2024
Viewed by 825
Abstract
Aircraft wake vortices are serious threats to aviation safety. The Pulsed Coherent Doppler Lidar (PCDL) has been widely used in the observation of aircraft wake vortices due to its advantages of high spatial-temporal resolution and high precision. However, the post-processing algorithms require significant [...] Read more.
Aircraft wake vortices are serious threats to aviation safety. The Pulsed Coherent Doppler Lidar (PCDL) has been widely used in the observation of aircraft wake vortices due to its advantages of high spatial-temporal resolution and high precision. However, the post-processing algorithms require significant computing resources, which cannot achieve the real-time detection of a wake vortex (WV). This paper presents an improved Convolutional Neural Network (CNN) method for WV locating and grading based on PCDL data to avoid the influence of unstable ambient wind fields on the localization and classification results of WV. Typical WV cases are selected for analysis, and the WV locating and grading models are validated on different test sets. The consistency of the analytical algorithm and the CNN algorithm is verified. The results indicate that the improved CNN method achieves satisfactory recognition accuracy with higher efficiency and better robustness, especially in the case of strong turbulence, where the CNN method recognizes the wake vortex while the analytical method cannot. The improved CNN method is expected to be applied to optimize the current aircraft spacing criteria, which is promising in terms of aviation safety and economic benefit improvement. Full article
(This article belongs to the Special Issue Computer Vision-Based Methods and Tools in Remote Sensing)
Show Figures

Figure 1

Figure 1
<p>Information of experiments for wake vortex observation based on PCDL. (<b>a</b>) Location of ZUUU and ZSQD. (<b>b</b>) Sketch map of wake vortex observation experiments at ZUUU. (<b>c</b>) Sketch map of wake vortex observation experiments at ZSQD. (<b>d</b>) Lidar position and scanning mode of wake vortex observation experiments at ZUUU. (<b>e</b>) Lidar position and scanning mode of wake vortex observation experiments at ZSQD.</p>
Full article ">Figure 2
<p>Flow chart of the main methodology of aircraft wake vortex locating and grading algorithm.</p>
Full article ">Figure 3
<p>Images of wake vortex observed at ZSQD at 00:47 on 27 April 2020, LST, where the brighter color represents a larger value. (<b>a</b>) The pseudo-color images of radial velocity (RV). (<b>b</b>) The pseudo-color images of spectrum width (SW); (<b>c</b>) The gray-scale images of RV. (<b>d</b>) The gray-scale images of SW. (<b>e</b>) The stacked image of radial velocity and spectrum width (RV-SW).</p>
Full article ">Figure 4
<p>RV-SW images after marking the regions of the wake vortices with dashed boxes, where the brighter color represents a larger value. (<b>a</b>,<b>b</b>) RV-SW images with marked wake vortex regions at ZSQD. (<b>c</b>,<b>d</b>) RV-SW images with marked wake vortex regions at ZUUU.</p>
Full article ">Figure 5
<p>The structure of the WV locating model based on YOLO v4 with different color blocks representing different layers. The color blocks marked with * are the results of combining different layers, the structure of which are explained in the lower part of the diagram marked with *.</p>
Full article ">Figure 6
<p>The WV region data set after extracting the wake vortex feature from the RV-SW data set, where the brighter color represents a larger value. (<b>a</b>) Example of Grade 0. (<b>b</b>) Example of Grade 1. (<b>c</b>) Example of Grade 2. (<b>d</b>) Example of Grade 3.</p>
Full article ">Figure 7
<p>The structure of the deep learning model used for wake vortex grading based on WV region images with different color blocks representing different layers. The color blocks marked with * are the results of combining different layers, the structure of which are explained in the lower part of the diagram marked with *.</p>
Full article ">Figure 8
<p>The images and results of the wake vortex under stable meteorological condition. (<b>a</b>) Spectrum width image of the wake vortex. (<b>b</b>) Radial velocity image of the wake vortex. (<b>c</b>) Results of WV locating and grading models when applied on the wake vortex, where the brighter color represents a larger value.</p>
Full article ">Figure 9
<p>The images and results of the wake vortex under strong turbulence condition. (<b>a</b>) Spectrum width image of the wake vortex. (<b>b</b>) Radial velocity image of the wake vortex. (<b>c</b>) Results of horizontal location of the wake vortex when using the analytical algorithm, where the light blue line represents <math display="inline"><semantics> <mrow> <mi>S</mi> <mi>W</mi> <mo>_</mo> <mi>i</mi> <mi>n</mi> <mi>d</mi> </mrow> </semantics></math>, the light pink line represents <math display="inline"><semantics> <mrow> <mi>R</mi> <mi>V</mi> <mo>_</mo> <mi>i</mi> <mi>n</mi> <mi>d</mi> </mrow> </semantics></math>, and the black line represents <math display="inline"><semantics> <mrow> <mi>S</mi> <mi>R</mi> <mo>_</mo> <mi>i</mi> <mi>n</mi> <mi>d</mi> </mrow> </semantics></math>, the red pentagram represents the location of the vortex core as identified by the algorithm. (<b>d</b>) Results of WV locating and grading models when applied on the wake vortex, where the brighter color represents a larger value.</p>
Full article ">Figure 10
<p>The confusion matrix of the WV grading model.</p>
Full article ">Figure 11
<p>The value of EDR<sup>1/3</sup> of typical dates at ZUUU and ZSQD.</p>
Full article ">
12 pages, 3758 KiB  
Article
Fusion of Single and Integral Multispectral Aerial Images
by Mohamed Youssef and Oliver Bimber
Remote Sens. 2024, 16(4), 673; https://doi.org/10.3390/rs16040673 - 14 Feb 2024
Viewed by 1648
Abstract
An adequate fusion of the most significant salient information from multiple input channels is essential for many aerial imaging tasks. While multispectral recordings reveal features in various spectral ranges, synthetic aperture sensing makes occluded features visible. We present a first and hybrid (model- [...] Read more.
An adequate fusion of the most significant salient information from multiple input channels is essential for many aerial imaging tasks. While multispectral recordings reveal features in various spectral ranges, synthetic aperture sensing makes occluded features visible. We present a first and hybrid (model- and learning-based) architecture for fusing the most significant features from conventional aerial images with the ones from integral aerial images that are the result of synthetic aperture sensing for removing occlusion. It combines the environment’s spatial references with features of unoccluded targets that would normally be hidden by dense vegetation. Our method outperforms state-of-the-art two-channel and multi-channel fusion approaches visually and quantitatively in common metrics, such as mutual information, visual information fidelity, and peak signal-to-noise ratio. The proposed model does not require manually tuned parameters, can be extended to an arbitrary number and arbitrary combinations of spectral channels, and is reconfigurable for addressing different use cases. We demonstrate examples for search and rescue, wildfire detection, and wildlife observation. Full article
(This article belongs to the Special Issue Computer Vision-Based Methods and Tools in Remote Sensing)
Show Figures

Figure 1

Figure 1
<p>Airborne Optical Sectioning (AOS) principle (<b>a</b>): Registering and integrating multiple images captured along a synthetic aperture of size a while computationally focusing on focal plane F at distance h will defocus occluders O at distance o from F (with a point-spread of b) while focusing targets on F. Single RGB image, SRGB (<b>b</b>); integral thermal image, IT (<b>c</b>); and integral RGB image, IRGB (<b>d</b>) of dense forest captured with a square synthetic aperture area (30 m × 30 m) with 1 m × 3 m dense sampling while computationally focusing on the forest ground. All images show the same scene, at the same time, from the same pose. (<b>e</b>) Fused result from (<b>b</b>–<b>d</b>). Close-ups are in dashed boxes. (<b>f</b>) Fusion of IT and IRGB under dark light conditions—increasing exposure through integration. Original SRGB image in solid box. (<b>g</b>) SRGB and IT fusion of sparse forest—highlighting the bottoms of tree trunks in the thermal channel. Close-up in dashed box. (<b>h</b>) SRGB and color-coded IT fusion—revealing hot ground patches in the IT channel. Color-coded IT image in sloid boxes.</p>
Full article ">Figure 2
<p>Proposed hybrid fusion architecture. Multiple input channels (one basis channel that remains unmodified and provides spatial references, and an arbitrary number of additional channels from which salient features are extracted) are fused into one composite image. Each feature channel applies multiple model-based and learning-based feature extractors (unified filters and VGG-layers in our case). Variables refer to Equations (1)–(7).</p>
Full article ">Figure 3
<p>Comparison of fusion results with unified filter only, VGG only, and the combination of unified filter and VGG. Only in the latter case background noise and sampling artifacts can be removed efficiently and essential target features are enhanced. The input images (<span class="html-italic">SRGB</span>, <span class="html-italic">IT</span>, <span class="html-italic">IRGB</span>) show the same scene, at the same time, from the same pose.</p>
Full article ">Figure 4
<p>Comparison of our approach with several state-of-the-art two- and multi-channel image fusion techniques for various scenes in a search-and-rescue use case. Input channels (<span class="html-italic">SRGB</span>, <span class="html-italic">IT</span>, <span class="html-italic">IRGB</span>) indicated with a yellow solid box contain useful features to be fused. They show the same scene, at the same time, from the same pose. The data used for our experiments and details on how they were recorded can be found in [<a href="#B3-remotesensing-16-00673" class="html-bibr">3</a>]. All images, except for (<b>c</b>), have been brightness-increased by 25% for better visibility. References: Hui Li et al., 2018 [<a href="#B13-remotesensing-16-00673" class="html-bibr">13</a>], Park et al., 2023 [<a href="#B19-remotesensing-16-00673" class="html-bibr">19</a>], Zhao et al., 2023 [<a href="#B16-remotesensing-16-00673" class="html-bibr">16</a>], Liu et al., 2023 [<a href="#B21-remotesensing-16-00673" class="html-bibr">21</a>], and Jiayi Ma et al., 2022 [<a href="#B20-remotesensing-16-00673" class="html-bibr">20</a>].</p>
Full article ">Figure 5
<p>Comparison of our approach with the state of the-art for wildlife observation (<b>a</b>) and wildfire detection and monitoring (<b>b</b>,<b>c</b>) use cases. Input channels (<span class="html-italic">SRGB</span>, <span class="html-italic">IT</span>, <span class="html-italic">IRGB</span>) are indicated with a yellow solid box. The example in (<b>a</b>) shows results from nesting observations of breeding herons. The data are taken from [<a href="#B2-remotesensing-16-00673" class="html-bibr">2</a>]. The data used for the examples in (<b>b</b>,<b>c</b>) are taken from the FLAME 2 dataset [<a href="#B29-remotesensing-16-00673" class="html-bibr">29</a>], while structure-from-motion and multi-view stereo [<a href="#B30-remotesensing-16-00673" class="html-bibr">30</a>] was applied for pose estimation of each video frame. Integral thermal images (IT) are computed from single thermal video frames with [<a href="#B1-remotesensing-16-00673" class="html-bibr">1</a>]. The IT channel is color-coded in all examples. References: Park et al., 2023 [<a href="#B19-remotesensing-16-00673" class="html-bibr">19</a>], Zhao et al., 2023 [<a href="#B16-remotesensing-16-00673" class="html-bibr">16</a>], Liu et al., 2023 [<a href="#B21-remotesensing-16-00673" class="html-bibr">21</a>], Jiayi Ma et al., 2022 [<a href="#B20-remotesensing-16-00673" class="html-bibr">20</a>], and Hui Li et al., 2018 [<a href="#B13-remotesensing-16-00673" class="html-bibr">13</a>].</p>
Full article ">
18 pages, 9369 KiB  
Article
Quantifying the Loss of Coral from a Bleaching Event Using Underwater Photogrammetry and AI-Assisted Image Segmentation
by Kai L. Kopecky, Gaia Pavoni, Erica Nocerino, Andrew J. Brooks, Massimiliano Corsini, Fabio Menna, Jordan P. Gallagher, Alessandro Capra, Cristina Castagnetti, Paolo Rossi, Armin Gruen, Fabian Neyer, Alessandro Muntoni, Federico Ponchio, Paolo Cignoni, Matthias Troyer, Sally J. Holbrook and Russell J. Schmitt
Remote Sens. 2023, 15(16), 4077; https://doi.org/10.3390/rs15164077 - 18 Aug 2023
Cited by 7 | Viewed by 7889
Abstract
Detecting the impacts of natural and anthropogenic disturbances that cause declines in organisms or changes in community composition has long been a focus of ecology. However, a tradeoff often exists between the spatial extent over which relevant data can be collected, and the [...] Read more.
Detecting the impacts of natural and anthropogenic disturbances that cause declines in organisms or changes in community composition has long been a focus of ecology. However, a tradeoff often exists between the spatial extent over which relevant data can be collected, and the resolution of those data. Recent advances in underwater photogrammetry, as well as computer vision and machine learning tools that employ artificial intelligence (AI), offer potential solutions with which to resolve this tradeoff. Here, we coupled a rigorous photogrammetric survey method with novel AI-assisted image segmentation software in order to quantify the impact of a coral bleaching event on a tropical reef, both at an ecologically meaningful spatial scale and with high spatial resolution. In addition to outlining our workflow, we highlight three key results: (1) dramatic changes in the three-dimensional surface areas of live and dead coral, as well as the ratio of live to dead colonies before and after bleaching; (2) a size-dependent pattern of mortality in bleached corals, where the largest corals were disproportionately affected, and (3) a significantly greater decline in the surface area of live coral, as revealed by our approximation of the 3D shape compared to the more standard planar area (2D) approach. The technique of photogrammetry allows us to turn 2D images into approximate 3D models in a flexible and efficient way. Increasing the resolution, accuracy, spatial extent, and efficiency with which we can quantify effects of disturbances will improve our ability to understand the ecological consequences that cascade from small to large scales, as well as allow more informed decisions to be made regarding the mitigation of undesired impacts. Full article
(This article belongs to the Special Issue Computer Vision-Based Methods and Tools in Remote Sensing)
Show Figures

Figure 1

Figure 1
<p>Site location. (<b>a</b>) satellite image of geographical location of French Polynesia in the South Pacific Ocean; (<b>b</b>) satellite image of Moorea, French Polynesia with a box indicating the location of the study site; (<b>c</b>) the fore reef of Moorea near our study site after the bleaching event that occurred in April of 2019 (PC: A. Thurber).</p>
Full article ">Figure 2
<p>Schematic of workflow. (<b>a</b>) custom-designed mount for a photogrammetry target that can be screwed into an anchor mounted in the reef substrate (PC: K. Kopecky); (<b>b</b>) diver taking photographs of the reef with custom photogrammetry targets in place (PC: R. Honeycutt); (<b>c</b>) orthophotomosaic of a single reef plot before the bleaching event; (<b>d</b>) interactive AI-based segmentations using TagLab (bright pink shapes) inside of a designated working area (shaded square) on the orthophotomosaic; (<b>e</b>) zoomed-in view of fully automated annotations of live corals in TagLab; (<b>f</b>) an example live coral colony (left panel) that died after the bleaching event (right panel).</p>
Full article ">Figure 3
<p>Example of 3D model layers for a single reef plot and epoch. (<b>a</b>) 3D mesh model with shaded rendering; (<b>b</b>) 3D mesh model with ambient occlusion rendering; (<b>c</b>) digital elevation model (DEM); and (<b>d</b>) orthorectified photomosaic.</p>
Full article ">Figure 4
<p>The TagLab computer interface. A plot with automatic segmentations of live and dead coral before (<b>left</b>) and after (<b>right</b>) the bleaching event. The small images show a live coral colony (<b>left</b>, pink shading) that died as a result of a bleaching event (<b>right</b>, brown shading). White boxes and lines indicate where the example colony is located in each of the larger images. The panels on the far right display the various attributes of the plot and the annotated colonies. From top to bottom: total coverage of designated coral classes (i.e., live and dead coral); a data table showing all annotated colonies, co-registered through time; attributes of the selected colony (e.g., 2D planar area, approximated 3D surface area, and perimeter); and a map preview showing the portion of the entire orthophotomosaic that is currently displayed.</p>
Full article ">Figure 5
<p>Changes in live and dead coral from before to after bleaching. (<b>a</b>) The proportions of the total number of colonies (summed across replicate plots) in each year that were either alive (gray) or dead (black). Total number of colonies before bleaching: N = 4306; total number of colonies after bleaching: N = 4015. (<b>b</b>) Total approximated 3D surface areas (m<sup>2</sup>) of live coral (gray) and dead coral (black). Triangles indicate means, error bars are ±1 SE, lines connect means through time, and small dots are plot (replicate) totals.</p>
Full article ">Figure 6
<p>The mean number (±1 SE; N = 5 plots) of dead coral colonies per plot in three size classes (approximated 3D surface area in cm<sup>2</sup>), before (2018, dark blue) and after (2019, light blue) the bleaching event.</p>
Full article ">Figure 7
<p>Comparison of estimates of live coral using approximated 3D surface area (m<sup>2</sup>, triangles) and 2D planar area (m<sup>2</sup>, squares) in 2018 (pre-bleaching, dark blue) and 2019 (post-bleaching, light blue) (plots as replicates). Triangles and squares represent means, error bars are ±1 SE, and lines connect observations through time.</p>
Full article ">
25 pages, 4426 KiB  
Article
Converging Channel Attention Mechanisms with Multilayer Perceptron Parallel Networks for Land Cover Classification
by Xiangsuo Fan, Xuyang Li, Chuan Yan, Jinlong Fan, Lin Chen and Nayi Wang
Remote Sens. 2023, 15(16), 3924; https://doi.org/10.3390/rs15163924 - 8 Aug 2023
Cited by 2 | Viewed by 1198
Abstract
This paper proposes a network structure called CAMP-Net, which considers the problem that traditional deep learning algorithms are unable to manage the pixel information of different bands, resulting in poor differentiation of feature representations of different categories and causing classification overfitting. CAMP-Net is [...] Read more.
This paper proposes a network structure called CAMP-Net, which considers the problem that traditional deep learning algorithms are unable to manage the pixel information of different bands, resulting in poor differentiation of feature representations of different categories and causing classification overfitting. CAMP-Net is a parallel network that, firstly, enhances the interaction of local information of bands by grouping the spectral nesting of the band information and then proposes a parallel processing model. One branch is responsible for inputting the features, normalized difference vegetation index (NDVI) and normalized difference water index (NDWI) band information generated by grouped nesting into the ViT framework, and enhancing the interaction and information flow between different channels in the feature map by adding the channel attention mechanism to realize the expressive capability of the feature map. The other branch assists the network’s ability to enhance the extraction of different feature channels by designing a multi-layer perceptron network based on the utilization of the feature channels. Finally, the classification results are obtained by fusing the features obtained by the channel attention mechanism with those obtained by the MLP to achieve pixel-level multispectral image classification. In this study, the application of the algorithm was carried out in the feature distribution of South County, Yiyang City, Hunan Province, and the experiments were conducted based on 10 m Sentinel-2 multispectral RS images. The experimental results show that the overall accuracy of the algorithm proposed in this paper is 99.00% and the transformer (ViT) is 95.81%, while the performance of the algorithm in the Sentinel-2 dataset was greatly improved for the transformer. The transformer shows a huge improvement, which provides research value for developing a land cover classification algorithm for remote sensing images. Full article
(This article belongs to the Special Issue Computer Vision-Based Methods and Tools in Remote Sensing)
Show Figures

Figure 1

Figure 1
<p>Location of the study area and Sentinel-2 remote sensing image (6 October 2021).</p>
Full article ">Figure 2
<p>Schematic diagram of the CAMP-Net architecture for multispectral image classification tasks. Where * stands for multiplication.</p>
Full article ">Figure 3
<p>Schematic diagram of the structure of the multilayer perceptron based on feature channels: (<b>a</b>) MLP block. (<b>b</b>) MLP branch structure.</p>
Full article ">Figure 4
<p>Channel Attention Module.</p>
Full article ">Figure 5
<p>Plot of the classification results of CAMP-Net using different combinations of modules for the study area dataset: (<b>a</b>) Image. (<b>b</b>) ViT. (<b>c</b>) CAMP-Net (NDVI + NDWI). (<b>d</b>) CAMP-Net (MLP). (<b>e</b>) CAMP-Net (CA). (<b>f</b>) CAMP-Net (MLP + NDVI + NDWI). (<b>g</b>) CAMP-Net (CA + NDVI + NDWI). (<b>h</b>) CAMP-Net (CA + MLP). (<b>i</b>) CAMP-Net (CA + MLP + NDVI + NDWI).</p>
Full article ">Figure 6
<p>Plots of the results of different proportions of training samples: (<b>a</b>) 10%. (<b>b</b>) 20%. (<b>c</b>) 30%. (<b>d</b>) 40%. (<b>e</b>) 50%. (<b>f</b>) 60%. (<b>g</b>) 70%. (<b>h</b>) 80%. (<b>i</b>) 90%.</p>
Full article ">Figure 7
<p>ViT results in different bands: (<b>a</b>) 4 bands. (<b>b</b>) 4 bands + VNIR. (<b>c</b>) 4 bands + SWIR. CAMP-Net results in different bands. (<b>d</b>) 4 bands. (<b>e</b>) 4 bands + VNIR. (<b>f</b>) 4 bands + SWIR.</p>
Full article ">Figure 8
<p>Classification results obtained by different models on South County data: (<b>a</b>) Image. (<b>b</b>) SVM. (<b>c</b>) KNN. (<b>d</b>) RF. (<b>e</b>) CNN. (<b>f</b>) RNN. (<b>g</b>) Transformer (ViT). (<b>h</b>) SF. (<b>i</b>) CAMP-Net.</p>
Full article ">Figure 9
<p>Classification results obtained by different models on Xiangyin County dataset: (<b>a</b>) Image. (<b>b</b>) SVM. (<b>c</b>) KNN. (<b>d</b>) RF. (<b>e</b>) CNN. (<b>f</b>) RNN. (<b>g</b>) Transformer (ViT). (<b>h</b>) SF. (<b>i</b>) CAMP-Net.</p>
Full article ">Figure 10
<p>True color maps of the three images: (<b>a</b>) 2018, (<b>b</b>) 2020, and (<b>c</b>) 2022; spatial distribution of features in the three images: (<b>d</b>) 2018, (<b>e</b>) 2020, and (<b>f</b>) 2022.</p>
Full article ">Figure 11
<p>Spatial distribution of pond in three images: (<b>a</b>) 2018, (<b>b</b>) 2020, and (<b>c</b>) 2022; spatial distribution of rapeseed in three images (<b>d</b>) 2018, (<b>e</b>) 2020, and (<b>f</b>) 2022; spatial distribution of vegetable in three images (<b>g</b>) 2018, (<b>h</b>) 2020, and (<b>i</b>) 2022.</p>
Full article ">Figure 12
<p>Dynamics of rapeseed in 2018–2020 (<b>a</b>), 2020–2022 (<b>b</b>), and 2018–2022 (<b>c</b>). Dynamics of vegetable in 2018–2020 (<b>d</b>), 2020–2022 (<b>e</b>), and 2018–2022 (<b>f</b>).</p>
Full article ">
23 pages, 28102 KiB  
Article
Multi-SUAV Collaboration and Low-Altitude Remote Sensing Technology-Based Image Registration and Change Detection Network of Garbage Scattered Areas in Nature Reserves
by Kai Yan, Yaxin Dong, Yang Yang and Lin Xing
Remote Sens. 2022, 14(24), 6352; https://doi.org/10.3390/rs14246352 - 15 Dec 2022
Cited by 1 | Viewed by 1620
Abstract
Change detection is an important task in remote sensing image processing and analysis. However, due to position errors and wind interference, bi-temporal low-altitude remote sensing images collected by SUAVs often suffer from different viewing angles. The existing methods need to use an independent [...] Read more.
Change detection is an important task in remote sensing image processing and analysis. However, due to position errors and wind interference, bi-temporal low-altitude remote sensing images collected by SUAVs often suffer from different viewing angles. The existing methods need to use an independent registration network for registration before change detection, which greatly reduces the integrity and speed of the task. In this work, we propose an end-to-end network architecture RegCD-Net to address change detection problems in the bi-temporal SUAVs’ low-altitude remote sensing images. We utilize global and local correlations to generate an optical flow pyramid and realize image registration through layer-by-layer optical flow fields. Then we use a nested connection to combine the rich semantic information in deep layers of the network and the precise location information in the shallow layers and perform deep supervision through the combined attention module to finally achieve change detection in bi-temporal images. We apply this network to the task of change detection in the garbage-scattered areas of nature reserves and establish a related dataset. Experimental results show that our RegCD-Net outperforms several state-of-the-art CD methods with more precise change edge representation, relatively few parameters, fast speed, and better integration without additional registration networks. Full article
(This article belongs to the Special Issue Computer Vision-Based Methods and Tools in Remote Sensing)
Show Figures

Figure 1

Figure 1
<p>The architecture of the RegCD network.</p>
Full article ">Figure 2
<p>The architecture of the optical flow pyramid with global and local correlations.</p>
Full article ">Figure 3
<p>The architecture of up-sampling nested connection.</p>
Full article ">Figure 4
<p>The structure of the convolution block.</p>
Full article ">Figure 5
<p>The architecture of CGAM.</p>
Full article ">Figure 6
<p>The multi-UAV collaboration platform.</p>
Full article ">Figure 7
<p>The procedure of dataset generation.</p>
Full article ">Figure 8
<p>The overview of the dataset. Optical flow field color is encoded with the same method used in FlowNet [<a href="#B25-remotesensing-14-06352" class="html-bibr">25</a>].</p>
Full article ">Figure 9
<p>Visualization results of different methods on our dataset. Four groups of different bi-temporal images are marked with (<b>a</b>–<b>d</b>). Different colors are used for a better view, which is white for true positive, black for true negative, red for false positive and green for false negative. The change maps in the first row of each group only contain false positives and false negatives for better edge error representation.</p>
Full article ">Figure 10
<p>The registration results in <math display="inline"><semantics> <mi>β</mi> </semantics></math> = <math display="inline"><semantics> <mrow> <mn>1</mn> <mo>×</mo> <msup> <mn>10</mn> <mrow> <mo>−</mo> <mn>3</mn> </mrow> </msup> </mrow> </semantics></math>. <math display="inline"><semantics> <msub> <mi>W</mi> <mi>i</mi> </msub> </semantics></math> denotes different optical flow pyramid levels.</p>
Full article ">
19 pages, 15010 KiB  
Article
DSM Generation from Multi-View High-Resolution Satellite Images Based on the Photometric Mesh Refinement Method
by Benchao Lv, Jianchen Liu, Ping Wang and Muhammad Yasir
Remote Sens. 2022, 14(24), 6259; https://doi.org/10.3390/rs14246259 - 10 Dec 2022
Cited by 2 | Viewed by 2492
Abstract
Automatic reconstruction of DSMs from satellite images is a hot issue in the field of photogrammetry. Nowadays, most state-of-the-art pipelines produce 2.5D products. In order to solve some shortcomings of traditional algorithms and expand the means of updating digital surface models, a DSM [...] Read more.
Automatic reconstruction of DSMs from satellite images is a hot issue in the field of photogrammetry. Nowadays, most state-of-the-art pipelines produce 2.5D products. In order to solve some shortcomings of traditional algorithms and expand the means of updating digital surface models, a DSM generation method based on variational mesh refinement of satellite stereo image pairs to recover 3D surfaces from coarse input is proposed. Specifically, the initial coarse mesh is constructed first and the geometric features of the generated 3D mesh model are then optimized by using the information of the original images, while the 3D mesh subdivision is constrained by combining the image’s texture information and projection information, with subdivision optimization of the mesh model finally achieved. The results of this method are compared qualitatively and quantitatively with those of the commercial software PCI and the SGM method. The experimental results show that the generated 3D digital surface has clearer edge contours, more refined planar textures, and sufficient model accuracy to match well with the actual conditions of the ground surface, proving the effectiveness of the method. The method is advantageous for conducting research on true 3D products in complex urban areas and can generate complete DSM products with the input of rough meshes, thus indicating it has some development prospects. Full article
(This article belongs to the Special Issue Computer Vision-Based Methods and Tools in Remote Sensing)
Show Figures

Graphical abstract

Graphical abstract
Full article ">Figure 1
<p>Flow chart of the mesh refinement method.</p>
Full article ">Figure 2
<p>Reprojection diagram between a reference image and reprojected image.</p>
Full article ">Figure 3
<p>Diagram of the Laplace operation for vertices in a mesh.</p>
Full article ">Figure 4
<p>Diagram of the resolution between image point and object point.</p>
Full article ">Figure 5
<p>Simulation of a ray.</p>
Full article ">Figure 6
<p>Multi-scale refinement strategies from low to high.</p>
Full article ">Figure 7
<p>Results of the two views: (<bold>a</bold>) original image pair, (<bold>b</bold>) coarse mesh generated by the pair, (<bold>c</bold>) the result after refinement (grayscale rendering), (<bold>d</bold>) coarse mesh (color rendering), (<bold>e</bold>) refined result (color rendering).</p>
Full article ">Figure 8
<p>Results with different conditions (<bold>a</bold>) result with higher projection area threshold (<bold>b</bold>) result with low texture complexity threshold (<bold>c</bold>) result with suitable parameters.</p>
Full article ">Figure 9
<p>WorldView-3 image: (<bold>a</bold>) test site 1, (<bold>b</bold>) test site 2.</p>
Full article ">Figure 10
<p>Ground truth LiDAR DSM data: (<bold>a</bold>) test site 1, (<bold>b</bold>) test site 2.</p>
Full article ">Figure 11
<p>Results of refinement: (<bold>a</bold>) test site 1, (<bold>b</bold>) test site 2.</p>
Full article ">Figure 12
<p>Local details of refinement and the PCI result (test site 1).</p>
Full article ">Figure 13
<p>Results for test site 1 with different methods: (<bold>a</bold>) input, (<bold>b</bold>) refinement, (<bold>c</bold>) SGM, (<bold>d</bold>) PCI.</p>
Full article ">Figure 14
<p>Details of local areas (<bold>a</bold>) Low-rise buildings (<bold>b</bold>) Point cloud of Low-rise buildings (<bold>c</bold>) Single building (<bold>d</bold>) Point cloud of Single building (<bold>e</bold>) Mid-rise buildings (<bold>f</bold>) Point cloud of Mid-rise buildings (<bold>g</bold>) Dense building area (<bold>h</bold>) Point cloud of Dense building area.</p>
Full article ">
Back to TopTop