Search Results (534)

The purpose of this research is to contribute to the development of approaches for the classification and segmentation of various gastrointestinal (GI) cancer diseases, such as dyed lifted polyps, dyed resection margins, esophagitis, normal cecum, normal pylorus, normal Z line, polyps, and ulcerative colitis. This research is relevant and essential because of the current challenges related to the absence of efficient diagnostic tools for early diagnostics of GI cancers, which are fundamental for improving the diagnosis of these common diseases. To address the above challenges, we propose a new hybrid segmentation model, U-MaskNet, which is a combination of U-Net and Mask R-CNN models. Here, U-Net is utilized for pixel-wise classification and Mask R-CNN for instance segmentation, together forming a solution for classifying and segmenting GI cancer. The Kvasir dataset, which includes 8000 endoscopic images of various GI cancers, is utilized to validate the proposed methodology. The experimental results clearly demonstrated that the novel proposed model provided superior segmentation compared to other well-known models, such as DeepLabv3+, FCN, and DeepMask, as well as improved classification performance compared to state-of-the-art (SOTA) models, including LeNet-5, AlexNet, VGG-16, ResNet-50, and the Inception Network. The quantitative analysis revealed that our proposed model outperformed the other models, achieving a precision of 98.85%, recall of 98.49%, and F1 score of 98.68%. Additionally, the novel model achieved a Dice coefficient of 94.35% and IoU of 89.31%. Consequently, the developed model increased the accuracy and reliability in detecting and segmenting GI cancer, and it was proven that the proposed model can potentially be used for improving the diagnostic process and, consequently, patient care in the clinical environment. This work highlights the benefits of integrating the U-Net and Mask R-CNN models, opening the way for further research in medical image segmentation. Full article

Unmanned aerial systems (UASs) are increasingly playing a crucial role in earthquake emergency response and disaster assessment due to their ease of operation, mobility, and low cost. However, post-earthquake scenes are complex, with many forms of damaged buildings. UAS imagery has a high spatial resolution, but the resolution is inconsistent between different flight missions. These factors make it challenging for existing methods to accurately identify individual damaged buildings in UAS images from different scenes, resulting in coarse segmentation masks that are insufficient for practical application needs. To address these issues, this paper proposed DB-Transfiner, a building damage instance segmentation method for post-earthquake UAS imagery based on the Mask Transfiner network. This method primarily employed deformable convolution in the backbone network to enhance adaptability to collapsed buildings of arbitrary shapes. Additionally, it used an enhanced bidirectional feature pyramid network (BiFPN) to integrate multi-scale features, improving the representation of targets of various sizes. Furthermore, a lightweight Transformer encoder has been used to process edge pixels, enhancing the efficiency of global feature extraction and the refinement of target edges. We conducted experiments on post-disaster UAS images collected from the 2022 Luding earthquake with a surface wave magnitude (Ms) of 6.8 in the Sichuan Province of China. The results demonstrated that the average precisions (AP) of DB-Transfiner, ${AP}_{box}$ and ${AP}_{seg}$, are 56.42% and 54.85%, respectively, outperforming all other comparative methods. Our model improved the original model by 5.00% and 4.07% in ${AP}_{box}$ and ${AP}_{seg}$, respectively. Importantly, the ${AP}_{seg}$ of our model was significantly higher than the state-of-the-art instance segmentation model Mask R-CNN, with an increase of 9.07%. In addition, we conducted applicability testing, and the model achieved an average correctness rate of 84.28% for identifying images from different scenes of the same earthquake. We also applied the model to the Yangbi earthquake scene and found that the model maintained good performance, demonstrating a certain level of generalization capability. This method has high accuracy in identifying and assessing damaged buildings after earthquakes and can provide critical data support for disaster loss assessment. Full article

When monitoring crop growth using top-down images of the plant canopies, leaves in agricultural fields appear very dense and significantly overlap each other. Moreover, the image can be affected by external conditions such as background environment and light intensity, impacting the effectiveness of image segmentation. To address the challenge of segmenting dense and overlapping plant leaves under natural lighting conditions, this study employed a Bilayer Convolutional Network (BCNet) method for accurate leaf segmentation across various lighting environments. The major contributions of this study are as follows: (1) Utilized Fully Convolutional Object Detection (FCOS) for plant leaf detection, incorporating ResNet-50 with the Convolutional Block Attention Module (CBAM) and Feature Pyramid Network (FPN) to enhance Region of Interest (RoI) feature extraction from canopy top-view images. (2) Extracted the sub-region of the RoI based on the position of the detection box, using this region as input for the BCNet, ensuring precise segmentation. (3) Utilized instance segmentation of canopy top-view images using BCNet, improving segmentation accuracy. (4) Applied the Varifocal Loss Function to improve the classification loss function in FCOS, leading to better performance metrics. The experimental results on cucumber canopy top-view images captured in glass greenhouse and plastic greenhouse environments show that our method is highly effective. For cucumber leaves at different growth stages and under various lighting conditions, the Precision, Recall and Average Precision (AP) metrics for object recognition are 97%, 94% and 96.57%, respectively. For instance segmentation, the Precision, Recall and Average Precision (AP) metrics are 87%, 83% and 84.71%, respectively. Our algorithm outperforms commonly used deep learning algorithms such as Faster R-CNN, Mask R-CNN, YOLOv4 and PANet, showcasing its superior capability in complex agricultural settings. The results of this study demonstrate the potential of our method for accurate recognition and segmentation of highly overlapping leaves in diverse agricultural environments, significantly contributing to the application of deep learning algorithms in smart agriculture. Full article

This study proposes a Generative Adversarial Network (GAN)-based method for virtual furniture replacement within indoor scenes. The proposed method addresses the challenge of accurately positioning new furniture in an indoor space by combining image reconstruction with geometric matching through combining spatial transformer networks and GANs. The system leverages deep learning architectures like Mask R-CNN for executing image segmentation and generating masks, and it employs DeepLabv3+, EdgeConnect algorithms, and ST-GAN networks for carrying out virtual furniture replacement. With the proposed system, furniture shoppers can obtain a virtual shopping experience, providing an easier way to understand the aesthetic effects of furniture rearrangement without putting in effort to physically move furniture. The proposed system has practical applications in the furnishing industry and interior design practices, providing a cost-effective and efficient alternative to physical furniture replacement. The results indicate that the proposed method achieves accurate positioning of new furniture in indoor scenes with minimal distortion or displacement. The proposed system is limited to 2D front-view images of furniture and indoor scenes. Future work would involve synthesizing 3D scenes and expanding the system to replace furniture images photographed from different angles. This would enhance the efficiency and practicality of the proposed system for virtual furniture replacement in indoor scenes. Full article

Object detection architectures struggle to detect small objects across applications including remote sensing and autonomous vehicles. Specifically, for unmanned aerial vehicles, poor detection of small objects directly limits this technology’s applicability. Objects both appear smaller than they are in large-scale images captured in aerial imagery and are represented by reduced information in high-altitude imagery. This paper presents a new architecture, CR-CNN, which predicts independent regions of interest from two unique prediction branches within the first stage of the network: a conventional R-CNN convolutional backbone and an hourglass backbone. Utilizing two independent sources within the first stage, our approach leads to an increase in successful predictions of regions that contain smaller objects. Anchor-based methods such as R-CNNs also utilize less than half the number of small objects compared to larger ones during training due to the poor intersection over union (IoU) scores between the generated anchors and the groundtruth—further reducing their performance on small objects. Therefore, we also propose artificially inflating the IoU of smaller objects during training using a simple, size-based Gaussian multiplier—leading to an increase in the quantity of small objects seen per training cycle based on an increase in the number of anchor–object pairs during training. This architecture and training strategy led to improved detection overall on two challenging aerial-based datasets heavily composed of small objects while predicting fewer false positives compared to Mask R-CNN. These results suggest that while new and unique architectures will continue to play a part in advancing the field of object detection, the training methodologies and strategies used will also play a valuable role. Full article

This study addresses the issue of inaccurate and error-prone grading judgments in luffa plug seedlings. A new Seg-FL seedling segmentation model is proposed as an extension of the YOLOv5s-Seg model. The small leaves of early-stage luffa seedlings are liable to be mistaken for impurities in the plug trays. To address this issue, cross-scale connections and weighted feature fusion are introduced in order to integrate feature information from different levels, thereby improving the recognition and segmentation accuracy of seedlings or details by refining the PANet structure. To address the ambiguity of seedling edge information during segmentation, an efficient channel attention module is incorporated to enhance the network’s focus on seedling edge information and suppress irrelevant features, thus sharpening the model’s focus on luffa seedlings. By optimizing the CIoU loss function, the calculation of overlapping areas, center point distances, and aspect ratios between predicted and ground truth boxes is preserved, thereby accelerating the convergence process and reducing the computational resource requirements on edge devices. The experimental results demonstrate that the proposed model attains a mean average precision of 97.03% on a self-compiled luffa plug seedling dataset, representing a 6.23 percentage point improvement over the original YOLOv5s-Seg. Furthermore, compared to the YOLACT++, FCN, and Mask R-CNN segmentation models, the improved model displays increases in [email protected] of 12.93%, 13.73%, and 10.53%, respectively, and improvements in precision of 15.73%, 16.93%, and 13.33%, respectively. This research not only validates the viability of the enhanced model for luffa seedling grading but also provides tangible technical support for the automation of grading in agricultural production. Full article

Mapping the distribution of living and dead trees in forests, particularly in ecologically fragile areas where forests serve as crucial ecological environments, is essential for assessing forest health, carbon storage capacity, and biodiversity. Convolutional neural networks, including Mask R-CNN, can assist in rapid and accurate forest monitoring. In this study, Mask R-CNN was employed to detect the crowns of Casuarina equisetifolia and to distinguish between live and dead trees in the Pingtan Comprehensive Pilot Zone, Fujian, China. High-resolution images of five plots were obtained using a multispectral Unmanned Aerial Vehicle. Six band combinations and derivatives, RGB, RGB-digital surface model (DSM), Multispectral, Multispectral-DSM, Vegetation Index, and Vegetation-Index-DSM, were used for tree crown detection and classification of live and dead trees. Five-fold cross-validation was employed to divide the manually annotated dataset of 21,800 live trees and 7157 dead trees into training and validation sets, which were used for training and validating the Mask R-CNN models. The results demonstrate that the RGB band combination achieved the most effective detection performance for live trees (average F1 score = 74.75%, IoU = 70.85%). The RGB–DSM combination exhibited the highest accuracy for dead trees (average F1 score = 71.16%, IoU = 68.28%). The detection performance for dead trees was lower than for live trees, which may be due to the similar spectral features across the images and the similarity of dead trees to the background, resulting in false identification. For the simultaneous detection of living and dead trees, the RGB combination produced the most promising results (average F1 score = 74.18%, IoU = 69.8%). It demonstrates that the Mask R-CNN model can achieve promising results for the detection of live and dead trees. Our study could provide forest managers with detailed information on the forest condition, which has the potential to improve forest management. Full article

Logs require multiple verifications to ensure accurate volume and quantity measurements. Log end detection is a crucial step in measuring log volume and counting logs. Currently, this task primarily relies on the Mask R-CNN instance segmentation model. However, the Feature Pyramid Network (FPN) in Mask R-CNN may compromise accuracy due to feature redundancy during multi-scale fusion, particularly with small objects. Moreover, counting logs in a single image is challenging due to their large size and stacking. To address the above issues, we propose an improved log segmentation model based on Cascade Mask R-CNN. This method uses ResNet for multi-scale feature extraction and integrates a hierarchical Convolutional Block Attention Module (CBAM) to refine feature weights and enhance object emphasis. Then, a Region Proposal Network (RPN) is employed to generate log segmentation proposals. Finally, combined with Deep SORT, the model tracks log ends in video streams and counts the number of logs in the stack. Experiments demonstrate the effectiveness of our method, achieving an average precision ($AP$) of 82.3, $A{P}_s$ of 75.3 for small, $A{P}_m$ of 70.9 for medium, and $A{P}_l$ of 86.2 for large objects. These results represent improvements of 1.8%, 3.7%, 2.6%, and 1.4% over Mask R-CNN, respectively. The detection rate reached 98.6%, with a counting accuracy of 95%. Compared to manually measured volumes, our method shows a low error rate of 4.07%. Full article

A probabilistic estimation model for forest biomass using unmanned aerial vehicle (UAV) photography was developed. We utilized a machine-learning-based object detection algorithm, a mask region-based convolutional neural network (Mask R-CNN), to detect trees in aerial photographs. Subsequently, Bayesian regression was used to calibrate the model based on an allometric model using the estimated crown diameter (CD) obtained from aerial photographs and analyzed the diameter at breast height (DBH) data acquired through terrestrial laser scanning. The F1 score of the Mask R-CNN for individual tree detection was 0.927. Moreover, CD estimation using the Mask R-CNN was acceptable (rRMSE = 10.17%). Accordingly, the probabilistic DBH estimation model was successfully calibrated using Bayesian regression. A predictive distribution accurately predicted the validation data, with 98.6% and 56.7% of the data being within the 95% and 50% prediction intervals, respectively. Furthermore, the estimated uncertainty of the probabilistic model was more practical and reliable compared to traditional ordinary least squares (OLS). Our model can be applied to estimate forest biomass at the individual tree level. Particularly, the probabilistic approach of this study provides a benefit for risk assessments. Additionally, since the workflow is not interfered by the tree canopy, it can effectively estimate forest biomass in dense canopy conditions. Full article

The circumpolar Taiga–Tundra Ecotone significantly influences the feedback mechanism of global climate change. Achieving large-scale individual tree crown (ITC) extraction in the transition zone is crucial for estimating vegetation biomass in the transition zone and studying plants’ response to climate change. This study employed aerial images and airborne LiDAR data covering several typical transitional zone regions in northern Finland to explore the ITC delineation method based on deep learning. First, this study developed an improved multi-scale ITC delineation method to enable the semi-automatic assembly of the ITC sample collection. This approach led to the creation of an individual tree dataset containing over 20,000 trees in the transitional zone. Then, this study explored the ITC delineation method using the Mask R-CNN model. The accuracies of the Mask R-CNN model were compared with two traditional ITC delineation methods: the improved multi-scale ITC delineation method and the local maxima clustering method based on point cloud distribution. For trees with a height greater than 1.3 m, the Mask R-CNN model achieved an overall recall rate ($A_r$) of 96.60%. Compared to the two conventional ITC delineation methods, the $A_r$ of Mask R-CNN showed an increase of 1.99 and 5.52 points in percentage, respectively, indicating that the Mask R-CNN model can significantly improve the accuracy of ITC delineation. These results highlight the potential of Mask R-CNN in extracting low trees with relatively small crowns in transitional zones using high-resolution aerial imagery and low-density airborne point cloud data for the first time. Full article

Background: A leading cause of emergency abdominal surgery, appendicitis is a common condition affecting millions of people worldwide. Automatic and accurate segmentation of the appendix from medical imaging is a challenging task, due to its small size, variability in shape, and proximity to other anatomical structures. Methods: In this study, we propose a backbone-enriched Mask R-CNN architecture (MaskAppendix) on the Detectron platform, enhanced with Gradient-weighted Class Activation Mapping (Grad-CAM), for precise appendix segmentation on computed tomography (CT) scans. In the proposed MaskAppendix deep learning model, ResNet101 network is used as the backbone. By integrating Grad-CAM into the MaskAppendix network, our model improves feature localization, allowing it to better capture subtle variations in appendix morphology. Results: We conduct extensive experiments on a dataset of abdominal CT scans, demonstrating that our method achieves state-of-the-art performance in appendix segmentation, outperforming traditional segmentation techniques in terms of both accuracy and robustness. In the automatic segmentation of the appendix region in CT slices, a DSC score of 87.17% was achieved with the proposed approach, and the results obtained have the potential to improve clinical diagnostic accuracy. Conclusions: This framework provides an effective tool for aiding clinicians in the diagnosis of appendicitis and other related conditions, reducing the potential for diagnostic errors and enhancing clinical workflow efficiency. Full article

As the main area for photosynthesis in trees, the canopy absorbs a large amount of carbon dioxide and plays an irreplaceable role in regulating the carbon cycle in the atmosphere and mitigating climate change. Therefore, monitoring the growth of the canopy is crucial. However, traditional field investigation methods are often limited by time-consuming and labor-intensive methods, as well as limitations in coverage, which may result in incomplete and inaccurate assessments. In response to the challenges encountered in the application of tree crown segmentation algorithms, such as adhesion between individual tree crowns and insufficient generalization ability of the algorithm, this study proposes an improved algorithm based on Mask R-CNN (Mask Region-based Convolutional Neural Network), which identifies irregular edges of tree crowns in RGB images obtained from drones. Firstly, it optimizes the backbone network by improving it to ResNeXt and embedding the SENet (Squeeze-and-Excitation Networks) module to enhance the model’s feature extraction capability. Secondly, the BiFPN-CBAM module is introduced to enable the model to learn and utilize features more effectively. Finally, it optimizes the mask loss function to the Boundary-Dice loss function to further improve the tree crown segmentation effect. In this study, TCSNet also incorporated the concept of panoptic segmentation, achieving the coherent and consistent segmentation of tree crowns throughout the entire scene through fine tree crown boundary recognition and integration. TCSNet was tested on three datasets with different geographical environments and forest types, namely artificial forests, natural forests, and urban forests, with artificial forests performing the best. Compared with the original algorithm, on the artificial forest dataset, the precision increased by 6.6%, the recall rate increased by 1.8%, and the F₁-score increased by 4.2%, highlighting its potential and robustness in tree detection and segmentation. Full article

The recently developed high-throughput system for cell spheroid generation (SpheroWell) is a promising technology for cost- and time-efficient in vitro analysis of, for example, chondrogenic differentiation. It is a compartmental growth surface where spheroids develop from a cell monolayer by self-assembling and aggregation. In order to automatize the analysis of spheroids, we aimed to develop imaging software and improve the localization of cell compartments and fully formed spheroids. Our workflow provides automated detection and localization of spheroids in different formation stages within Petri dishes based on images created with a low-budget camera imaging setup. This automated detection enables a fast and inexpensive analysis workflow by processing a stack of images within a short period of time, which is essential for the extraction of early readout parameters. Our workflow combines image processing algorithms and deep learning-based image localization/segmentation methods like Mask R-CNN and Unet++. These methods are refined by an evolution strategy for automated grid detection, which is able to improve the overall segmentation and classification quality. Besides the already pre-trained neural networks and predefined image processing parameters, our evolution-based post-processing provides the required adaptability for our workflow to deliver a consistent and reproducible quality. This is especially important due to the use of a low-budget imaging setup with various light conditions. The to-be-detected objects of the three different stages show improved results using our evolutionary post-processing for monolayer and starting aggregation with Dice coefficients of 0.7301 and 0.8562, respectively, compared with the raw scores of 0.2879 and 0.8187. The Dice coefficient of the fully formed spheroids in both cases is 0.8829. With our algorithm, we provide automated analyses of cell spheroid by self-assembling in SpheroWell dishes, even if the images are created using a low-budget camera setup. Full article

Agriculture stands as the cornerstone of Africa’s economy, supporting over 60% of the continent’s labor force. Despite its significance, the quality assessment of agricultural products remains a challenging task, particularly at a large scale, consuming valuable time and resources. The African plum is an agricultural fruit that is widely consumed across West and Central Africa but remains underrepresented in AI research. In this paper, we collected a dataset of 2892 African plum samples from fields in Cameroon representing the first dataset of its kind for training AI models. The dataset contains images of plums annotated with quality grades. We then trained and evaluated various state-of-the-art object detection and image classification models, including YOLOv5, YOLOv8, YOLOv9, Fast R-CNN, Mask R-CNN, VGG-16, DenseNet-121, MobileNet, and ResNet, on this African plum dataset. Our experimentation resulted in mean average precision scores ranging from 88.2% to 89.9% and accuracies between 86% and 91% for the object detection models and the classification models, respectively. We then performed model pruning to reduce model sizes while preserving performance, achieving up to 93.6% mean average precision and 99.09% accuracy after pruning YOLOv5, YOLOv8 and ResNet by 10–30%. We deployed the high-performing YOLOv8 system in a web application, offering an accessible AI-based quality assessment tool tailored for African plums. To the best of our knowledge, this represents the first such solution for assessing this underrepresented fruit, empowering farmers with efficient tools. Our approach integrates agriculture and AI to fill a key gap. Full article

Demand and market value for pennywort largely depend on the quality of the leaves, which can be affected by various ambient environment or fertigation variables during cultivation. Although early detection of defects in pennywort leaves would enable growers to take quick action, conventional manual detection is laborious and time consuming as well as subjective. Therefore, the objective of this study was to develop an automatic leaf defect detection algorithm for pennywort plants grown under controlled environment conditions, using machine vision and deep learning techniques. Leaf images were captured from pennywort plants grown in an ebb-and-flow hydroponic system under fluorescent light conditions in a controlled plant factory environment. Physically or biologically damaged leaves (e.g., curled, creased, discolored, misshapen, or brown spotted) were classified as defective leaves. Images were annotated using an online tool, and Mask R-CNN models were implemented with the integrated attention mechanisms, convolutional block attention module (CBAM) and coordinate attention (CA) and compared for improved image feature extraction. Transfer learning was employed to train the model with a smaller dataset, effectively reducing processing time. The improved models demonstrated significant advancements in accuracy and precision, with the CA-augmented model achieving the highest metrics, including a mean average precision (mAP) of 0.931 and an accuracy of 0.937. These enhancements enabled more precise localization and classification of leaf defects, outperforming the baseline Mask R-CNN model in complex visual recognition tasks. The final model was robust, effectively distinguishing defective leaves in challenging scenarios, making it highly suitable for applications in precision agriculture. Future research can build on this modeling framework, exploring additional variables to identify specific leaf abnormalities at earlier growth stages, which is crucial for production quality assurance. Full article