Journal of Imaging

13 pages, 3398 KiB

Open AccessArticle

On the Relationship between Corneal Biomechanics, Macrostructure, and Optical Properties

by Francisco J. Ávila, Maria Concepción Marcellán and Laura Remón

J. Imaging 2021, 7(12), 280; https://doi.org/10.3390/jimaging7120280 - 18 Dec 2021

Cited by 4 | Viewed by 2920

Optical properties of the cornea are responsible for correct vision; the ultrastructure allows optical transparency, and the biomechanical properties govern the shape, elasticity, or stiffness of the cornea, affecting ocular integrity and intraocular pressure. Therefore, the optical aberrations, corneal transparency, structure, and biomechanics [...] Read more.

Optical properties of the cornea are responsible for correct vision; the ultrastructure allows optical transparency, and the biomechanical properties govern the shape, elasticity, or stiffness of the cornea, affecting ocular integrity and intraocular pressure. Therefore, the optical aberrations, corneal transparency, structure, and biomechanics play a fundamental role in the optical quality of human vision, ocular health, and refractive surgery outcomes. However, the inter-relationships of those properties are not yet reported at a macroscopic scale within the hierarchical structure of the cornea. This work explores the relationships between the biomechanics, structure, and optical properties (corneal aberrations and optical density) at a macro-structural level of the cornea through dual Placido–Scheimpflug imaging and air-puff tonometry systems in a healthy young adult population. Results showed correlation between optical transparency, corneal macrostructure, and biomechanics, whereas corneal aberrations and in particular spherical terms remained independent. A compensation mechanism for the spherical aberration is proposed through corneal shape and biomechanics. Full article

(This article belongs to the Special Issue Biomechanical Techniques for Biomedical Imaging)

► Show Figures

Figure 1

20 pages, 1564 KiB

Open AccessArticle

A Robust Tensor-Based Submodule Clustering for Imaging Data Using

l_{\frac{1}{2}}

Regularization and Simultaneous Noise Recovery via Sparse and Low Rank Decomposition Approach

by Jobin Francis, Baburaj Madathil, Sudhish N. George and Sony George

J. Imaging 2021, 7(12), 279; https://doi.org/10.3390/jimaging7120279 - 17 Dec 2021

Cited by 4 | Viewed by 3257

Abstract

The massive generation of data, which includes images and videos, has made data management, analysis, information extraction difficult in recent years. To gather relevant information, this large amount of data needs to be grouped. Real-life data may be noise corrupted during data collection [...] Read more.

The massive generation of data, which includes images and videos, has made data management, analysis, information extraction difficult in recent years. To gather relevant information, this large amount of data needs to be grouped. Real-life data may be noise corrupted during data collection or transmission, and the majority of them are unlabeled, allowing for the use of robust unsupervised clustering techniques. Traditional clustering techniques, which vectorize the images, are unable to keep the geometrical structure of the images. Hence, a robust tensor-based submodule clustering method based on

l_{\frac{1}{2}}

regularization with improved clustering capability is formulated. The

l_{\frac{1}{2}}

induced tensor nuclear norm (TNN), integrated into the proposed method, offers better low rankness while retaining the self-expressiveness property of submodules. Unlike existing methods, the proposed method employs a simultaneous noise removal technique by twisting the lateral image slices of the input data tensor into frontal slices and eliminates the noise content in each image, using the principles of the sparse and low rank decomposition technique. Experiments are carried out over three datasets with varying amounts of sparse, Gaussian and salt and pepper noise. The experimental results demonstrate the superior performance of the proposed method over the existing state-of-the-art methods. Full article

► Show Figures

Figure 1

13 pages, 6982 KiB

Open AccessArticle

Word Spotting as a Service: An Unsupervised and Segmentation-Free Framework for Handwritten Documents

by Konstantinos Zagoris, Angelos Amanatiadis and Ioannis Pratikakis

J. Imaging 2021, 7(12), 278; https://doi.org/10.3390/jimaging7120278 - 17 Dec 2021

Cited by 3 | Viewed by 3179

Abstract

Word spotting strategies employed in historical handwritten documents face many challenges due to variation in the writing style and intense degradation. In this paper, a new method that permits efficient and effective word spotting in handwritten documents is presented that relies upon document-oriented [...] Read more.

Word spotting strategies employed in historical handwritten documents face many challenges due to variation in the writing style and intense degradation. In this paper, a new method that permits efficient and effective word spotting in handwritten documents is presented that relies upon document-oriented local features that take into account information around representative keypoints and a matching process that incorporates a spatial context in a local proximity search without using any training data. The method relies on a document-oriented keypoint and feature extraction, along with a fast feature matching method. This enables the corresponding methodological pipeline to be both effectively and efficiently employed in the cloud so that word spotting can be realised as a service in modern mobile devices. The effectiveness and efficiency of the proposed method in terms of its matching accuracy, along with its fast retrieval time, respectively, are shown after a consistent evaluation of several historical handwritten datasets. Full article

(This article belongs to the Special Issue Document Recognition in the Cultural Heritage: Methods and Applications)

► Show Figures

Figure 1

10 pages, 3143 KiB

Open AccessArticle

Liouville Integrability in a Four-Dimensional Model of the Visual Cortex

by Ivan Galyaev and Alexey Mashtakov

J. Imaging 2021, 7(12), 277; https://doi.org/10.3390/jimaging7120277 - 17 Dec 2021

Cited by 1 | Viewed by 2088

Abstract

We consider a natural extension of the Petitot–Citti–Sarti model of the primary visual cortex. In the extended model, the curvature of contours is taken into account. The occluded contours are completed via sub-Riemannian geodesics in the four-dimensional space M of positions, orientations, and [...] Read more.

We consider a natural extension of the Petitot–Citti–Sarti model of the primary visual cortex. In the extended model, the curvature of contours is taken into account. The occluded contours are completed via sub-Riemannian geodesics in the four-dimensional space M of positions, orientations, and curvatures. Here,

M = R^{2} \times SO (2) \times R

models the configuration space of neurons of the visual cortex. We study the problem of sub-Riemannian geodesics on M via methods of geometric control theory. We prove complete controllability of the system and the existence of optimal controls. By application of the Pontryagin maximum principle, we derive a Hamiltonian system that describes the geodesics. We obtain the explicit parametrization of abnormal extremals. In the normal case, we provide three functionally independent first integrals. Numerical simulations indicate the existence of one more first integral that results in Liouville integrability of the system. Full article

(This article belongs to the Special Issue Mathematical Modeling of Human Vision and Its Application to Image Processing)

► Show Figures

Figure 1

Figure 1
In the four-dimensional model of the visual cortex, an occluded contour is completed via the planar projection of a sub-Riemannian length-minimizer in the space <math display="inline"><semantics> <mrow> <msup> <mrow> <mi mathvariant="double-struck">R</mi> </mrow> <mn>2</mn> </msup> <mo>×</mo> <mi>SO</mi> <mrow> <mo>(</mo> <mn>2</mn> <mo>)</mo> </mrow> <mo>×</mo> <mi mathvariant="double-struck">R</mi> <mo>=</mo> <mi>M</mi> <mo>∋</mo> <mrow> <mo>(</mo> <mi>x</mi> <mo>,</mo> <mi>y</mi> <mo>,</mo> <mi>θ</mi> <mo>,</mo> <mi>k</mi> <mo>)</mo> </mrow> </mrow> </semantics></math> of positions, orientations, and curvatures. In the left column, we show an example of the image with partially occluded contours and the complete image. In the right column, we show a trajectory that satisfies the given boundary conditions. The curvature is visualized as its reciprocal—the radius of the osculating circle. Full article ">Figure 2
Orbits of the Poincare map in the space <math display="inline"><semantics> <mrow> <mo>(</mo> <mi>α</mi> <mo>,</mo> <msub> <mi>h</mi> <mn>3</mn> </msub> <mo>,</mo> <msub> <mi>h</mi> <mn>4</mn> </msub> <mo>)</mo> </mrow> </semantics></math> are formed by intersection points of the transversal hyperspace <math display="inline"><semantics> <mrow> <mi>k</mi> <mo>=</mo> <mn>0</mn> </mrow> </semantics></math> with trajectories close to the periodic one (red dot). Different orbits are depicted in different colors. Starting points are indicated for each trajectory. Full article ">

16 pages, 3707 KiB

Open AccessArticle

A Pipelined Tracer-Aware Approach for Lesion Segmentation in Breast DCE-MRI

by Antonio Galli, Stefano Marrone, Gabriele Piantadosi, Mario Sansone and Carlo Sansone

J. Imaging 2021, 7(12), 276; https://doi.org/10.3390/jimaging7120276 - 14 Dec 2021

Cited by 7 | Viewed by 2840

Abstract

The recent spread of Deep Learning (DL) in medical imaging is pushing researchers to explore its suitability for lesion segmentation in Dynamic Contrast-Enhanced Magnetic-Resonance Imaging (DCE-MRI), a complementary imaging procedure increasingly used in breast-cancer analysis. Despite some promising proposed solutions, we argue that [...] Read more.

The recent spread of Deep Learning (DL) in medical imaging is pushing researchers to explore its suitability for lesion segmentation in Dynamic Contrast-Enhanced Magnetic-Resonance Imaging (DCE-MRI), a complementary imaging procedure increasingly used in breast-cancer analysis. Despite some promising proposed solutions, we argue that a “naive” use of DL may have limited effectiveness as the presence of a contrast agent results in the acquisition of multimodal 4D images requiring thorough processing before training a DL model. We thus propose a pipelined approach where each stage is intended to deal with or to leverage a peculiar characteristic of breast DCE-MRI data: the use of a breast-masking pre-processing to remove non-breast tissues; the use of Three-Time-Points (3TP) slices to effectively highlight contrast agent time course; the application of a motion-correction technique to deal with patient involuntary movements; the leverage of a modified U-Net architecture tailored on the problem; and the introduction of a new “Eras/Epochs” training strategy to handle the unbalanced dataset while performing a strong data augmentation. We compared our pipelined solution against some literature works. The results show that our approach outperforms the competitors by a large margin (+9.13% over our previous solution) while also showing a higher generalization ability. Full article

(This article belongs to the Special Issue Bio-Medical Multimodal Methods for Diagnosis, Prognosis, and Outcome Prediction)

► Show Figures

Figure 1

Figure 1
A breast DCE-MRI study and an illustrative time intensity curve. On the left (a), the 4D multimodal volume, with the signal intensity variations reflecting the contrast agent flow over time. On the right (b), the corresponding (illustrative) time intensity curve for a single voxel. The vertical line separates the pre-contrast (early) from post-contrast injection instants. Full article ">Figure 2
Number of studies published between 2001 and the first ten months of 2021 on Google Scholar filtered by “Breast Lesion Segmentation” as topic keyword. Full article ">Figure 3
The proposed pipelined segmentation schema: in the first stage, all the extraneous tissues and background air are removed; in the second stage, a motion-correction technique is used to register each post-contrast 3D-volume to the pre-contrast one; in the third stage, for each slice, the corresponding 3TP slice (a three-channel image) is generated by concatenating homologous slices identified by the three-time points defined in [<a href="#B11-jimaging-07-00276" class="html-bibr">11</a>]; finally, in the fourth stage, each lesion is segmented by using a modified U-Net to produce the final lesion binary mask. Full article ">Figure 4
The considered 3TP U-Net architecture. On the left, the encoding side gradually decreases the spatial resolution while increasing the feature size. On the right, the decoding side gradually increases the spatial resolution from the inner embedding to the final output mask. Dotted lines and capital L highlight the network levels, with L5 being the deeper one. Big grey arrows illustrate the sharing of the cropped feature map, within the same layer, from the encoding to the decoding side, happening during the up-sampling. Compared to the classical U-Net architecture, the considered model varies for the use of batch normalization after each ReLU activation (in both encoding and decoding sides), for the use of zero padding, and for the use of a single output map. Full article ">Figure 5
Eras/epochs training schema: there are as many eras i as needed by the network to converge; there are as many epochs n as the number of chunks; finally, within each epoch n, k batches are built by using the samples from the corresponding chunk <math display="inline"><semantics> <msub> <mi>c</mi> <mi>n</mi> </msub> </semantics></math>. Full article ">Figure 6
Violin plots for patient-wise (dots in the image) segmentation performance for all the deep-learning-based approaches considered in this work. For a fair comparison, all the plots were generated by setting the kernel density bandwidth to 10. Full article ">

16 pages, 12889 KiB

Open AccessArticle

Characterisation of Single-Phase Fluid-Flow Heterogeneity Due to Localised Deformation in a Porous Rock Using Rapid Neutron Tomography

by Maddi Etxegarai, Erika Tudisco, Alessandro Tengattini, Gioacchino Viggiani, Nikolay Kardjilov and Stephen A. Hall

J. Imaging 2021, 7(12), 275; https://doi.org/10.3390/jimaging7120275 - 14 Dec 2021

Cited by 5 | Viewed by 2825

Abstract

The behaviour of subsurface-reservoir porous rocks is a central topic in the resource engineering industry and has relevant applications in hydrocarbon, water production, and CO

_{2}

sequestration. One of the key open issues is the effect of deformation on the hydraulic properties of [...] Read more.

The behaviour of subsurface-reservoir porous rocks is a central topic in the resource engineering industry and has relevant applications in hydrocarbon, water production, and CO

_{2}

sequestration. One of the key open issues is the effect of deformation on the hydraulic properties of the host rock and, specifically, in saturated environments. This paper presents a novel full-field data set describing the hydro-mechanical properties of porous geomaterials through in situ neutron and X-ray tomography. The use of high-performance neutron imaging facilities such as CONRAD-2 (Helmholtz-Zentrum Berlin) allows the tracking of the fluid front in saturated samples, making use of the differential neutron contrast between “normal” water and heavy water. To quantify the local hydro-mechanical coupling, we applied a number of existing image analysis algorithms and developed an array of bespoke methods to track the water front and calculate the 3D speed maps. The experimental campaign performed revealed that the pressure-driven flow speed decreases, in saturated samples, in the presence of pre-existing low porosity heterogeneities and compactant shear-bands. Furthermore, the observed complex mechanical behaviour of the samples and the associated fluid flow highlight the necessity for 3D imaging and analysis. Full article

(This article belongs to the Special Issue Recent Advances in Image-Based Geotechnics)

► Show Figures

Figure 1

18 pages, 3219 KiB

Open AccessArticle

A System for Real-Time, Online Mixed-Reality Visualization of Cardiac Magnetic Resonance Images

by Dominique Franson, Andrew Dupuis, Vikas Gulani, Mark Griswold and Nicole Seiberlich

J. Imaging 2021, 7(12), 274; https://doi.org/10.3390/jimaging7120274 - 14 Dec 2021

Cited by 6 | Viewed by 3817

Abstract

Image-guided cardiovascular interventions are rapidly evolving procedures that necessitate imaging systems capable of rapid data acquisition and low-latency image reconstruction and visualization. Compared to alternative modalities, Magnetic Resonance Imaging (MRI) is attractive for guidance in complex interventional settings thanks to excellent soft tissue [...] Read more.

Image-guided cardiovascular interventions are rapidly evolving procedures that necessitate imaging systems capable of rapid data acquisition and low-latency image reconstruction and visualization. Compared to alternative modalities, Magnetic Resonance Imaging (MRI) is attractive for guidance in complex interventional settings thanks to excellent soft tissue contrast and large fields-of-view without exposure to ionizing radiation. However, most clinically deployed MRI sequences and visualization pipelines exhibit poor latency characteristics, and spatial integration of complex anatomy and device orientation can be challenging on conventional 2D displays. This work demonstrates a proof-of-concept system linking real-time cardiac MR image acquisition, online low-latency reconstruction, and a stereoscopic display to support further development in real-time MR-guided intervention. Data are acquired using an undersampled, radial trajectory and reconstructed via parallelized through-time radial generalized autocalibrating partially parallel acquisition (GRAPPA) implemented on graphics processing units. Images are rendered for display in a stereoscopic mixed-reality head-mounted display. The system is successfully tested by imaging standard cardiac views in healthy volunteers. Datasets comprised of one slice (46 ms), two slices (92 ms), and three slices (138 ms) are collected, with the acquisition time of each listed in parentheses. Images are displayed with latencies of 42 ms/frame or less for all three conditions. Volumetric data are acquired at one volume per heartbeat with acquisition times of 467 ms and 588 ms when 8 and 12 partitions are acquired, respectively. Volumes are displayed with a latency of 286 ms or less. The faster-than-acquisition latencies for both planar and volumetric display enable real-time 3D visualization of the heart. Full article

(This article belongs to the Special Issue The Application of Imaging Technology in Medical Intervention and Surgery)

► Show Figures

Figure 1

Figure 1
Concept illustration of the real-time, online mixed-reality visualization system. MR data are collected using an undersampled, radial trajectory for rapid acquisition. Real-time image reconstruction is performed using a parallelized implementation of through-time radial GRAPPA. The final images are rendered in a mixed-reality headset for intuitive display. Full article ">Figure 2
Schematic of data flow within the online acquisition, reconstruction, and mixed-reality rendering system. A dedicated image reconstruction computer was introduced into the network containing the scanner, the scanner control computer, and the scanner measurement computer via a switch and an Ethernet cable. A second network was set up using a router to contain the reconstruction computer and the mixed-reality headset. Two configurations were possible. In the first, reconstructed data are sent directly to a self-contained HMD capable of handling the entire rendering pipeline. In the second, a dedicated rendering workstation is used. The reconstruction and rendering computers are connected via Ethernet cables, and rendered frame data are transferred wirelessly to the headset. Solid gray lines show component connections, and dashed red lines depict data transfer. Full article ">Figure 3
Schematic showing the relative timings of the acquisition, reconstruction, and rendering processes and their sub-steps for a two-slice planar scan. Each frame comprises two slices that are acquired sequentially. The removal of readout oversampling is performed on each line of data as it is acquired, as indicated by the diagonal bar across the sub-step rectangle. Data for the slice are buffered, followed by coil compression, GRAPPA, the NUFFT, and export of the images to the rendering system. The rendering processing includes receiving the image data over the TCP socket, parsing the data, and processing the data. Once all slices per frame are processed, as detected by an update flag, the images are rendered to the user on the next headset frame update. The net display latency is considered to be the time between when all of the data for one frame have been acquired and when that frame is displayed to the user. Note that all components of the system may be active at once, operating on different slices and/or frames of data concurrently. Full article ">Figure 4
Schematic showing the time required for multiple frames of data to pass through the system for three different dataset types: (top) single-slice planar, (middle) two-slice planar, and (bottom) eight-partition volumetric. Planar data are acquired continuously, while volumetric data are ECG-gated. Images are displayed to the user before completing acquisition of the following frame for all cases. Different colors represent different frames. Full article ">Figure 5
Operation of the full system at the MR scanner. (Left) A user wearing a mixed-reality headset sits at the scanner control computer before scanning a healthy volunteer lying in the scanner bore (red dashed circle). (Middle) During the scan, the user sees a multi-slice rendering of the volunteer’s heart in real-time. (Right) User’s view of the rendering. Note that what appears black in the image appears transparent to the user; the user sees the rendering within the natural environment. A video version of this figure is available in the <a href="#app1-jimaging-07-00274" class="html-app">Supplementary Materials</a>. Full article ">Figure 6
Multiple still frames from a user’s perspective of a volumetric scan through a mixed-reality headset are shown. The user moved around the rendering, changed the viewing angle, and moved toward and away from it. The stills were captured by a program in the headset that combines the rendering that is projected to the user with a video capture from the camera embedded in the headset. A video version of this figure is available in the <a href="#app1-jimaging-07-00274" class="html-app">Supplementary Materials</a>. Full article ">Figure 7
Example 2D panel display and corresponding renderings. (Top left) Three slices in short-axis and four-chamber views. Note that the dark bands across the ventricles in the four-chamber view are from saturation of the signal where the short-axis slices intersect. (Top right) Rendering of the slices in the correct spatial positions and orientations. (Bottom left) Panel display of eight partitions in a volumetric dataset centered over the left ventricle. (Bottom right) Rendering of the dataset. Full article ">

17 pages, 66075 KiB

Open AccessArticle

Reliable Estimation of Deterioration Levels via Late Fusion Using Multi-View Distress Images for Practical Inspection

by Keisuke Maeda, Naoki Ogawa, Takahiro Ogawa and Miki Haseyama

J. Imaging 2021, 7(12), 273; https://doi.org/10.3390/jimaging7120273 - 9 Dec 2021

Cited by 1 | Viewed by 2416

Abstract

This paper presents reliable estimation of deterioration levels via late fusion using multi-view distress images for practical inspection. The proposed method simultaneously solves the following two problems that are necessary to support the practical inspection. Since maintenance of infrastructures requires a high level [...] Read more.

This paper presents reliable estimation of deterioration levels via late fusion using multi-view distress images for practical inspection. The proposed method simultaneously solves the following two problems that are necessary to support the practical inspection. Since maintenance of infrastructures requires a high level of safety and reliability, this paper proposes a neural network that can generate an attention map from distress images and text data acquired during the inspection. Thus, deterioration level estimation with high interpretability can be realized. In addition, since multi-view distress images are taken for single distress during the actual inspection, it is necessary to estimate the final result from these images. Therefore, the proposed method integrates estimation results obtained from the multi-view images via the late fusion and can derive an appropriate result considering all the images. To the best of our knowledge, no method has been proposed to solve these problems simultaneously, and this achievement is the biggest contribution of this paper. In this paper, we confirm the effectiveness of the proposed method by conducting experiments using data acquired during the actual inspection. Full article

(This article belongs to the Special Issue Intelligent Media Processing)

► Show Figures

Figure 1

25 pages, 12342 KiB

Open AccessArticle

Methodology for the Automated Visual Detection of Bird and Bat Collision Fatalities at Onshore Wind Turbines

by Christof Happ, Alexander Sutor and Klaus Hochradel

J. Imaging 2021, 7(12), 272; https://doi.org/10.3390/jimaging7120272 - 9 Dec 2021

Cited by 2 | Viewed by 2884

Abstract

The number of collision fatalities is one of the main quantification measures for research concerning wind power impacts on birds and bats. Despite being integral in ongoing investigations as well as regulatory approvals, the state-of-the-art method for the detection of fatalities remains a [...] Read more.

The number of collision fatalities is one of the main quantification measures for research concerning wind power impacts on birds and bats. Despite being integral in ongoing investigations as well as regulatory approvals, the state-of-the-art method for the detection of fatalities remains a manual search by humans or dogs. This is expensive, time consuming and the efficiency varies greatly among different studies. Therefore, we developed a methodology for the automatic detection using visual/near-infrared cameras for daytime and thermal cameras for nighttime. The cameras can be installed in the nacelle of wind turbines and monitor the area below. The methodology is centered around software that analyzes the images in real time using pixel-wise and region-based methods. We found that the structural similarity is the most important measure for the decision about a detection. Phantom drop tests in the actual wind test field with the system installed on 75 m above the ground resulted in a sensitivity of 75.6% for the nighttime detection and 84.3% for the daylight detection. The night camera detected 2.47 false positives per hour using a time window designed for our phantom drop tests. However, in real applications this time window can be extended to eliminate false positives caused by nightly active animals. Excluding these from our data reduced the false positive rate to 0.05. The daylight camera detected 0.20 false positives per hour. Our proposed method has the advantages of being more consistent, more objective, less time consuming, and less expensive than manual search methods. Full article

(This article belongs to the Special Issue Visual Localization)

► Show Figures

Figure 1

21 pages, 2435 KiB

Open AccessArticle

Multi-Frequency Image Completion via a Biologically-Inspired Sub-Riemannian Model with Frequency and Phase

by Emre Baspinar

J. Imaging 2021, 7(12), 271; https://doi.org/10.3390/jimaging7120271 - 9 Dec 2021

Cited by 2 | Viewed by 2359

Abstract

We present a novel cortically-inspired image completion algorithm. It uses five-dimensional sub-Riemannian cortical geometry, modeling the orientation, spatial frequency and phase-selective behavior of the cells in the visual cortex. The algorithm extracts the orientation, frequency and phase information existing in a given two-dimensional [...] Read more.

We present a novel cortically-inspired image completion algorithm. It uses five-dimensional sub-Riemannian cortical geometry, modeling the orientation, spatial frequency and phase-selective behavior of the cells in the visual cortex. The algorithm extracts the orientation, frequency and phase information existing in a given two-dimensional corrupted input image via a Gabor transform and represents those values in terms of cortical cell output responses in the model geometry. Then, it performs completion via a diffusion concentrated in a neighborhood along the neural connections within the model geometry. The diffusion models the activity propagation integrating orientation, frequency and phase features along the neural connections. Finally, the algorithm transforms the diffused and completed output responses back to the two-dimensional image plane. Full article

(This article belongs to the Special Issue Mathematical Modeling of Human Vision and Its Application to Image Processing)

► Show Figures

Figure 1

30 pages, 5078 KiB

Open AccessArticle

A Fast and Accurate Approach to Multiple-Vehicle Localization and Tracking from Monocular Aerial Images

by Daniel Tøttrup, Stinus Lykke Skovgaard, Jonas le Fevre Sejersen and Rui Pimentel de Figueiredo

J. Imaging 2021, 7(12), 270; https://doi.org/10.3390/jimaging7120270 - 8 Dec 2021

Cited by 3 | Viewed by 2571

Abstract

In this work we present a novel end-to-end solution for tracking objects (i.e., vessels), using video streams from aerial drones, in dynamic maritime environments. Our method relies on deep features, which are learned using realistic simulation data, for robust object detection, segmentation and [...] Read more.

In this work we present a novel end-to-end solution for tracking objects (i.e., vessels), using video streams from aerial drones, in dynamic maritime environments. Our method relies on deep features, which are learned using realistic simulation data, for robust object detection, segmentation and tracking. Furthermore, we propose the use of rotated bounding-box representations, which are computed by taking advantage of pixel-level object segmentation, for improved tracking accuracy, by reducing erroneous data associations during tracking, when combined with the appearance-based features. A thorough set of experiments and results obtained in a realistic shipyard simulation environment, demonstrate that our method can accurately, and fast detect and track dynamic objects seen from a top-view. Full article

(This article belongs to the Special Issue Visual Localization)

► Show Figures

Figure 1

16 pages, 3799 KiB

Open AccessArticle

Brain Tumor Segmentation Based on Deep Learning’s Feature Representation

by Ilyasse Aboussaleh, Jamal Riffi, Adnane Mohamed Mahraz and Hamid Tairi

J. Imaging 2021, 7(12), 269; https://doi.org/10.3390/jimaging7120269 - 8 Dec 2021

Cited by 32 | Viewed by 5431

Abstract

Brain tumor is considered as one of the most serious causes of death in the world. Thus, it is very important to detect it as early as possible. In order to predict and segment the tumor, many approaches have been proposed. However, they [...] Read more.

Brain tumor is considered as one of the most serious causes of death in the world. Thus, it is very important to detect it as early as possible. In order to predict and segment the tumor, many approaches have been proposed. However, they suffer from different problems such as the necessity of the intervention of a specialist, the long required run-time and the choice of the appropriate feature extractor. To address these issues, we proposed an approach based on convolution neural network architecture aiming at predicting and segmenting simultaneously a cerebral tumor. The proposal was divided into two phases. Firstly, aiming at avoiding the use of the labeled image that implies a subject intervention of the specialist, we used a simple binary annotation that reflects the existence of the tumor or not. Secondly, the prepared image data were fed into our deep learning model in which the final classification was obtained; if the classification indicated the existence of the tumor, the brain tumor was segmented based on the feature representations generated by the convolutional neural network architectures. The proposed method was trained on the BraTS 2017 dataset with different types of gliomas. The achieved results show the performance of the proposed approach in terms of accuracy, precision, recall and Dice similarity coefficient. Our model showed an accuracy of 91% in tumor classification and a Dice similarity coefficient of 82.35% in tumor segmentation. Full article

(This article belongs to the Section Medical Imaging)

► Show Figures

Figure 1

13 pages, 3512 KiB

Open AccessArticle

A Reversible Data Hiding Method in Encrypted Images for Controlling Trade-Off between Hiding Capacity and Compression Efficiency

by Ryota Motomura, Shoko Imaizumi and Hitoshi Kiya

J. Imaging 2021, 7(12), 268; https://doi.org/10.3390/jimaging7120268 - 7 Dec 2021

Cited by 5 | Viewed by 3089

Abstract

In this paper, we propose a new framework for reversible data hiding in encrypted images, where both the hiding capacity and lossless compression efficiency are flexibly controlled. There exist two main purposes; one is to provide highly efficient lossless compression under a required [...] Read more.

In this paper, we propose a new framework for reversible data hiding in encrypted images, where both the hiding capacity and lossless compression efficiency are flexibly controlled. There exist two main purposes; one is to provide highly efficient lossless compression under a required hiding capacity, while the other is to enable us to extract an embedded payload from a decrypted image. The proposed method can decrypt marked encrypted images without data extraction and derive marked images. An original image is arbitrarily divided into two regions. Two different methods for reversible data hiding in encrypted images (RDH-EI) are used in our method, and each one is used for either region. Consequently, one region can be decrypted without data extraction and also losslessly compressed using image coding standards even after the processing. The other region possesses a significantly high hiding rate, around 1 bpp. Experimental results show the effectiveness of the proposed method in terms of hiding capacity and lossless compression efficiency. Full article

(This article belongs to the Special Issue Intelligent Media Processing)

► Show Figures

Figure 1

21 pages, 3963 KiB

Open AccessArticle

A Semi-Supervised Reduced-Space Method for Hyperspectral Imaging Segmentation

by Giacomo Aletti, Alessandro Benfenati and Giovanni Naldi

J. Imaging 2021, 7(12), 267; https://doi.org/10.3390/jimaging7120267 - 7 Dec 2021

Cited by 8 | Viewed by 3028

Abstract

The development of the hyperspectral remote sensor technology allows the acquisition of images with a very detailed spectral information for each pixel. Because of this, hyperspectral images (HSI) potentially possess larger capabilities in solving many scientific and practical problems in agriculture, biomedical, ecological, [...] Read more.

The development of the hyperspectral remote sensor technology allows the acquisition of images with a very detailed spectral information for each pixel. Because of this, hyperspectral images (HSI) potentially possess larger capabilities in solving many scientific and practical problems in agriculture, biomedical, ecological, geological, hydrological studies. However, their analysis requires developing specialized and fast algorithms for data processing, due the high dimensionality of the data. In this work, we propose a new semi-supervised method for multilabel segmentation of HSI that combines a suitable linear discriminant analysis, a similarity index to compare different spectra, and a random walk based model with a direct label assignment. The user-marked regions are used for the projection of the original high-dimensional feature space to a lower dimensional space, such that the class separation is maximized. This allows to retain in an automatic way the most informative features, lightening the successive computational burden. The part of the random walk is related to a combinatorial Dirichlet problem involving a weighted graph, where the nodes are the projected pixel of the original HSI, and the positive weights depend on the distances between these nodes. We then assign to each pixel of the original image a probability quantifying the likelihood that the pixel (node) belongs to some subregion. The computation of the spectral distance involves both the coordinates in a features space of a pixel and of its neighbors. The final segmentation process is therefore reduced to a suitable optimization problem coupling the probabilities from the random walker computation, and the similarity with respect the initially labeled pixels. We discuss the properties of the new method with experimental results carried on benchmark images. Full article

(This article belongs to the Special Issue Advances in Multi/Hyperspectral Imaging)

► Show Figures

Figure 1

33 pages, 3286 KiB

Open AccessReview

Off-The-Grid Variational Sparse Spike Recovery: Methods and Algorithms

by Bastien Laville, Laure Blanc-Féraud and Gilles Aubert

J. Imaging 2021, 7(12), 266; https://doi.org/10.3390/jimaging7120266 - 6 Dec 2021

Cited by 7 | Viewed by 3163

Abstract

Gridless sparse spike reconstruction is a rather new research field with significant results for the super-resolution problem, where we want to retrieve fine-scale details from a noisy and filtered acquisition. To tackle this problem, we are interested in optimisation under some prior, typically [...] Read more.

Gridless sparse spike reconstruction is a rather new research field with significant results for the super-resolution problem, where we want to retrieve fine-scale details from a noisy and filtered acquisition. To tackle this problem, we are interested in optimisation under some prior, typically the sparsity i.e., the source is composed of spikes. Following the seminal work on the generalised LASSO for measures called the Beurling-Lasso (BLASSO), we will give a review on the chief theoretical and numerical breakthrough of the off-the-grid inverse problem, as we illustrate its usefulness to the super-resolution problem in Single Molecule Localisation Microscopy (SMLM) through new reconstruction metrics and tests on synthetic and real SMLM data we performed for this review. Full article

(This article belongs to the Special Issue Inverse Problems and Imaging)

► Show Figures

Figure 1

20 pages, 2675 KiB

Open AccessReview

Using Inertial Sensors to Determine Head Motion—A Review

by Severin Ionut-Cristian and Dobrea Dan-Marius

J. Imaging 2021, 7(12), 265; https://doi.org/10.3390/jimaging7120265 - 6 Dec 2021

Cited by 22 | Viewed by 5421

Abstract

Human activity recognition and classification are some of the most interesting research fields, especially due to the rising popularity of wearable devices, such as mobile phones and smartwatches, which are present in our daily lives. Determining head motion and activities through wearable devices [...] Read more.

Human activity recognition and classification are some of the most interesting research fields, especially due to the rising popularity of wearable devices, such as mobile phones and smartwatches, which are present in our daily lives. Determining head motion and activities through wearable devices has applications in different domains, such as medicine, entertainment, health monitoring, and sports training. In addition, understanding head motion is important for modern-day topics, such as metaverse systems, virtual reality, and touchless systems. The wearability and usability of head motion systems are more technologically advanced than those which use information from a sensor connected to other parts of the human body. The current paper presents an overview of the technical literature from the last decade on state-of-the-art head motion monitoring systems based on inertial sensors. This study provides an overview of the existing solutions used to monitor head motion using inertial sensors. The focus of this study was on determining the acquisition methods, prototype structures, preprocessing steps, computational methods, and techniques used to validate these systems. From a preliminary inspection of the technical literature, we observed that this was the first work which looks specifically at head motion systems based on inertial sensors and their techniques. The research was conducted using four internet databases—IEEE Xplore, Elsevier, MDPI, and Springer. According to this survey, most of the studies focused on analyzing general human activity, and less on a specific activity. In addition, this paper provides a thorough overview of the last decade of approaches and machine learning algorithms used to monitor head motion using inertial sensors. For each method, concept, and final solution, this study provides a comprehensive number of references which help prove the advantages and disadvantages of the inertial sensors used to read head motion. The results of this study help to contextualize emerging inertial sensor technology in relation to broader goals to help people suffering from partial or total paralysis of the body. Full article

(This article belongs to the Special Issue 3D Human Understanding)

► Show Figures

Figure 1

16 pages, 1297 KiB

Open AccessArticle

Skeleton-Based Attention Mask for Pedestrian Attribute Recognition Network

by Sorn Sooksatra and Sitapa Rujikietgumjorn

J. Imaging 2021, 7(12), 264; https://doi.org/10.3390/jimaging7120264 - 4 Dec 2021

Cited by 3 | Viewed by 2807

Abstract

This paper presents an extended model for a pedestrian attribute recognition network utilizing skeleton data as a soft attention model to extract a local feature corresponding to a specific attribute. This technique helped keep valuable information surrounding the target area and handle the [...] Read more.

This paper presents an extended model for a pedestrian attribute recognition network utilizing skeleton data as a soft attention model to extract a local feature corresponding to a specific attribute. This technique helped keep valuable information surrounding the target area and handle the variation of human posture. The attention masks were designed to focus on the partial and the whole-body regions. This research utilized an augmented layer for data augmentation inside the network to reduce over-fitting errors. Our network was evaluated in two datasets (RAP and PETA) with various backbone networks (ResNet-50, Inception V3, and Inception-ResNet V2). The experimental result shows that our network improves overall classification performance with a mean accuracy of about 2–3% in the same backbone network, especially local attributes and various human postures. Full article

(This article belongs to the Special Issue Advances in Human Action Recognition Using Deep Learning)

► Show Figures

Figure 1

16 pages, 5518 KiB

Open AccessArticle

Micro- and Nano-Scales Three-Dimensional Characterisation of Softwood

by Alessandra Patera, Anne Bonnin and Rajmund Mokso

J. Imaging 2021, 7(12), 263; https://doi.org/10.3390/jimaging7120263 - 3 Dec 2021

Cited by 7 | Viewed by 2818

Abstract

Understanding the mechanical response of cellular biological materials to environmental stimuli is of fundamental importance from an engineering perspective in composites. To provide a deep understanding of their behaviour, an exhaustive analytical and experimental protocol is required. Attention is focused on softwood but [...] Read more.

Understanding the mechanical response of cellular biological materials to environmental stimuli is of fundamental importance from an engineering perspective in composites. To provide a deep understanding of their behaviour, an exhaustive analytical and experimental protocol is required. Attention is focused on softwood but the approach can be applied to a range of cellular materials. This work presents a new non-invasive multi-scale approach for the investigation of the hygro-mechanical behaviour of softwood. At the TOMCAT beamline of the Paul Scherrer Institute, in Switzerland, the swelling behaviour of softwood was probed at the cellular and sub-cellular scales by means of 3D high-resolution phase-contrast X-ray imaging. At the cellular scale, new findings in the anisotropic and reversible swelling behaviour of softwood and in the origin of swelling hysteresis of porous materials are explained from a mechanical perspective. However, the mechanical and moisture properties of wood highly depend on sub-cellular features of the wood cell wall, such as bordered pits, yielding local deformations during a full hygroscopic loading protocol. Full article

(This article belongs to the Special Issue X-ray Digital Radiography and Computed Tomography)

► Show Figures

Figure 1

12 pages, 940 KiB

Open AccessArticle

Optical to Planar X-ray Mouse Image Mapping in Preclinical Nuclear Medicine Using Conditional Adversarial Networks

by Eleftherios Fysikopoulos, Maritina Rouchota, Vasilis Eleftheriadis, Christina-Anna Gatsiou, Irinaios Pilatis, Sophia Sarpaki, George Loudos, Spiros Kostopoulos and Dimitrios Glotsos

J. Imaging 2021, 7(12), 262; https://doi.org/10.3390/jimaging7120262 - 3 Dec 2021

Cited by 1 | Viewed by 3536

Abstract

In the current work, a pix2pix conditional generative adversarial network has been evaluated as a potential solution for generating adequately accurate synthesized morphological X-ray images by translating standard photographic images of mice. Such an approach will benefit 2D functional molecular imaging techniques, such [...] Read more.

In the current work, a pix2pix conditional generative adversarial network has been evaluated as a potential solution for generating adequately accurate synthesized morphological X-ray images by translating standard photographic images of mice. Such an approach will benefit 2D functional molecular imaging techniques, such as planar radioisotope and/or fluorescence/bioluminescence imaging, by providing high-resolution information for anatomical mapping, but not for diagnosis, using conventional photographic sensors. Planar functional imaging offers an efficient alternative to biodistribution ex vivo studies and/or 3D high-end molecular imaging systems since it can be effectively used to track new tracers and study the accumulation from zero point in time post-injection. The superimposition of functional information with an artificially produced X-ray image may enhance overall image information in such systems without added complexity and cost. The network has been trained in 700 input (photography)/ground truth (X-ray) paired mouse images and evaluated using a test dataset composed of 80 photographic images and 80 ground truth X-ray images. Performance metrics such as peak signal-to-noise ratio (PSNR), structural similarity index measure (SSIM) and Fréchet inception distance (FID) were used to quantitatively evaluate the proposed approach in the acquired dataset. Full article

(This article belongs to the Special Issue SPECT and PET Imaging of Small Animals)

► Show Figures

Figure 1

Figure 1
Indicative optical image acquired using a conventional photographic sensor located in BIOEMTECH eye’s series radioisotope screening tools (a); Indicative X-ray image acquired in a prototype PET/SPECT X-ray system (c) used as ground truth; Aligned pair (b). Full article ">Figure 2
Aligned image pairs used for training. Photographic input image (left); Corresponding X-ray scan used as ground truth (right). Full article ">Figure 3
The pix2pix Generator’s training layout. The generator creates output image (y) from input image (x) and random noise vector (z) and improves its performance by receiving feedback from the discriminator, as well as regarding the degree of fakeness of the synthetic image (y) compared to the ground truth (r). Full article ">Figure 4
The cGAN discriminator’s training layout. The discriminator compares the input(x)/ground truth(r) pair of images and the input(x)/output(y) pair of images and outputs its guess about how realistic they look. The weights vector of the discriminator is then updated based on the classification error of the input/output pair (D fake Loss) and the input/target pair (D real Loss). Full article ">Figure 5
Training loss curves of the cross entropy, M.S.E. and Wasserstein distance loss function models. Full article ">Figure 6
Indicative “fake” X-ray images from the pix2pix trained network using different loss functions: Cross Entropy (3rd column); MSE (4th column); Wasserstein distance (5th column). The input photographic images and the corresponding ground truth images are presented in the first two columns. Full article ">Figure 7
<math display="inline"><semantics> <mrow> <msup> <mrow/> <mrow> <mn>99</mn> <mi>m</mi> </mrow> </msup> <mi>T</mi> <mi>c</mi> </mrow> </semantics></math>-MDP-labelled nuclear image of a healthy mouse fused with the optical image provided in the <math display="inline"><semantics> <mi>γ</mi> </semantics></math>-eye scintigraphic system (left) and the X-ray produced from the pix2pix trained network (right). Full article ">Figure 8
<math display="inline"><semantics> <mrow> <msup> <mrow/> <mn>18</mn> </msup> <mi>F</mi> </mrow> </semantics></math>-FDG nuclear image of a healthy mouse fused with the optical image provided in the <math display="inline"><semantics> <mi>β</mi> </semantics></math>-eye planar coincidence imaging system (left) and the X-ray produced from the pix2pix trained network (right). Full article ">

10 pages, 1766 KiB

Open AccessArticle

Copper and Trace Elements in Gallbladder form Patients with Wilson’s Disease Imaged and Determined by Synchrotron X-ray Fluorescence

by Wolf Osterode, Gerald Falkenberg and Fritz Wrba

J. Imaging 2021, 7(12), 261; https://doi.org/10.3390/jimaging7120261 - 3 Dec 2021

Cited by 4 | Viewed by 2943

Abstract

Investigations about suspected tissue alterations and the role of gallbladder in Wilson’s disease (WD)—an inherited genetic disease with impaired copper metabolism—are rare. Therefore, tissue from patients with genetically characterised WD was investigated by microscopic synchrotron X-ray fluorescence (µSRXRF). For two-dimensional imaging and quantification [...] Read more.

Investigations about suspected tissue alterations and the role of gallbladder in Wilson’s disease (WD)—an inherited genetic disease with impaired copper metabolism—are rare. Therefore, tissue from patients with genetically characterised WD was investigated by microscopic synchrotron X-ray fluorescence (µSRXRF). For two-dimensional imaging and quantification of elements, X-ray spectra were peak-fitted, and the net peak intensities were normalised to the intensity of the incoming monochromatic beam intensity. Concentrations were calculated by fundamental parameter-based program quant and external standardisation. Copper (Cu), zinc (Zn) and iron (Fe) along with sulphur (S) and phosphorus (P) mappings could be demonstrated in a near histological resolution. All these elements were increased compared to gallbladder tissue from controls. Cu and Zn and Fe in WD-GB were mostly found to be enhanced in the epithelium. We documented a significant linear relationship with Cu, Zn and sulphur. Concentrations of Cu/Zn were roughly 1:1 while S/Cu was about 100:1, depending on the selected areas for investigation. The significant linear relationship with Cu, Zn and sulphur let us assume that metallothioneins, which are sulphur-rich proteins, are increased too. Our data let us suggest that the WD gallbladder is the first in the gastrointestinal tract to reabsorb metals to prevent oxidative damage caused by metal toxicity. Full article

(This article belongs to the Special Issue X-ray Digital Radiography and Computed Tomography)

► Show Figures

Figure 1

Figure 1
Accumulated spectra after summation over all pixels with peaks of the investigated elements. Red colour: Control–GB. Black colour: Wilson’s Disease-GB. Full article ">Figure 2
(a–f) Element distribution in a control gallbladder. (a) Microscope recording of the investigated gallbladder tissue with epithelium (columnar cells), beneath the epithelium the laminar propria (connective tissue) and smooth muscular fibres for contraction. Adventitia (Adv): perimuscular connective tissue that is very dense and connects with the liver. (b–f) copper (Cu), iron (Fe), zinc (Zn), phosphorous (P) and sulphur (S) distribution. The histological section is about 30-µm apart from the investigated one for SRXRF. Full article ">Figure 3
Correlations between elements in ppm for C-GB (<a href="#jimaging-07-00261-f002" class="html-fig">Figure 2</a>). (a) Cu:Zn = 1.28; rs = 0.92. (b) S:Cu = 70; rs = 0.92. (c) Correlations between S:Zn = 79, rs = 0.88. For all p < 0.001. Full article ">Figure 4
Distribution of elements in a WD gallbladder. (a) copper (Cu), (b) zinc (Zn), (c) iron (Fe), (d) sulphur (S) and (e) phosphorus (P). Full article ">Figure 5
Correlations between elements for a WD-GB. (a) Cu/Zn = 1.28; rs = 0.94. (b) S/Cu = 119; rs = 0.91. (c) Correlations between S/Zn = 81, rs = 0.93. For all p < 0.001. Full article ">

13 pages, 16300 KiB

Open AccessArticle

HTR for Greek Historical Handwritten Documents

by Lazaros Tsochatzidis, Symeon Symeonidis, Alexandros Papazoglou and Ioannis Pratikakis

J. Imaging 2021, 7(12), 260; https://doi.org/10.3390/jimaging7120260 - 2 Dec 2021

Cited by 9 | Viewed by 3553

Abstract

Offline handwritten text recognition (HTR) for historical documents aims for effective transcription by addressing challenges that originate from the low quality of manuscripts under study as well as from several particularities which are related to the historical period of writing. In this paper, [...] Read more.

Offline handwritten text recognition (HTR) for historical documents aims for effective transcription by addressing challenges that originate from the low quality of manuscripts under study as well as from several particularities which are related to the historical period of writing. In this paper, the challenge in HTR is related to a focused goal of the transcription of Greek historical manuscripts that contain several particularities. To this end, in this paper, a convolutional recurrent neural network architecture is proposed that comprises octave convolution and recurrent units which use effective gated mechanisms. The proposed architecture has been evaluated on three newly created collections from Greek historical handwritten documents that will be made publicly available for research purposes as well as on standard datasets like IAM and RIMES. For evaluation we perform a concise study which shows that compared to state of the art architectures, the proposed one deals effectively with the challenging Greek historical manuscripts. Full article

(This article belongs to the Special Issue Computer Vision and Robotics for Cultural Heritage: Theory and Applications)

► Show Figures

Figure 1

10 pages, 2080 KiB

Open AccessArticle

Infrared Clinical Enamel Crack Detector Based on Silicon CCD and Its Application: A High-Quality and Low-Cost Option

by Yuchen Zheng, Min-Hee Oh, Woo-Sub Song, Ki-Hyun Kim, In-Hee Shin, Min-Seok Kim and Jin-Hyoung Cho

J. Imaging 2021, 7(12), 259; https://doi.org/10.3390/jimaging7120259 - 2 Dec 2021

Cited by 5 | Viewed by 3207

Abstract

Enamel cracks generated in the anterior teeth not only affect the function but also the aesthetics of the teeth. Chair-side tooth enamel crack detection is essential for clinicians to formulate treatment plans and prevent related dental disease. This study aimed to develop a [...] Read more.

Enamel cracks generated in the anterior teeth not only affect the function but also the aesthetics of the teeth. Chair-side tooth enamel crack detection is essential for clinicians to formulate treatment plans and prevent related dental disease. This study aimed to develop a dental imaging system using a near-IR light source to detect enamel cracks and to investigate the relationship between anterior enamel cracks and age in vivo. A total of 68 subjects were divided into three groups according to their age: young, middle, and elderly. Near-infrared radiation of 850 nm was used to identify enamel cracks in anterior teeth. The results of the quantitative examination showed that the number of enamel cracks on the teeth increased considerably with age. For the qualitative examination, the results indicated that there was no significant relationship between the severity of the enamel cracks and age. So, it can be concluded that the prevalence of anterior cracked tooth increased significantly with age in the young and middle age. The length of the anterior enamel cracks tended to increase with age too; however, this result was not significant. The silicon charge-coupled device (CCD) with a wavelength of 850 nm has a good performance in the detection of enamel cracks and has very good clinical practicability. Full article

(This article belongs to the Special Issue New Frontiers of Advanced Imaging in Dentistry)

► Show Figures

Figure 1

Figure 1
Device and method of detection. (<bold>a</bold>) Rubber tray, which could emit 850 nm Near-IR in the mouth to highlight the cracks. Red arrows indicate the LED illuminant, whose brightness can be adjusted from level 1 to 5, with 5 being the brightest. (<bold>b</bold>) Near-infrared dental and periodontal imaging device with a screen on the examiner’s side for image capturing and a CCD camera inside for image acquiring. Red arrow indicates a control lever, which is connected to the camera to adjust its direction during the detection process. (<bold>c</bold>) Schematic diagram of the patient position during detection. Full article ">Figure 2
The three most representative types of teeth in the three age groups. The red arrow points out the enamel cracks. The high contrast of the enamel cracks in the near-infrared image can be clearly seen. Full article ">Figure 3
Simulated images, intraoral photos, and 850 nm Near-IR images for indicating the differences among the classifications. Red arrows indicate the cracks. (<bold>a</bold>) A simulated Class 1 central incisor crack. (<bold>b</bold>) An intraoral photo of a Class 1 crack on the maxillary central incisor under natural light. (<bold>c</bold>) The image of the tooth in <xref ref-type="fig" rid="jimaging-07-00259-f003">Figure 3</xref>b under Near-IR. (<bold>d</bold>) A simulated Class 2 central incisor crack. (<bold>e</bold>) The intraoral photo of a Class 2 crack on the maxillary central incisor under natural light. (<bold>f</bold>) The image of the tooth in <xref ref-type="fig" rid="jimaging-07-00259-f003">Figure 3</xref>e under Near-IR. (<bold>g</bold>) A simulated Class 3 central incisor crack. (<bold>h</bold>) An intraoral photo of a Class 3 crack on the maxillary central incisor under natural light. (<bold>i</bold>) The image of the tooth in <xref ref-type="fig" rid="jimaging-07-00259-f003">Figure 3</xref>h under Near-IR. Full article ">Figure 4
The distribution of different tooth types and tooth classifications among the three groups. (<bold>a</bold>) Tooth type distribution. The value of the ordinate indicates the number of cracked teeth. (<bold>b</bold>) Enamel crack classification distribution. The value of the ordinate indicates the number of different classes of cracks. Full article ">

12 pages, 2722 KiB

Open AccessArticle

Abdominal Computed Tomography Imaging Findings in Hospitalized COVID-19 Patients: A Year-Long Experience and Associations Revealed by Explainable Artificial Intelligence

by Alice Scarabelli, Massimo Zilocchi, Elena Casiraghi, Pierangelo Fasani, Guido Giovanni Plensich, Andrea Alessandro Esposito, Elvira Stellato, Alessandro Petrini, Justin Reese, Peter Robinson, Giorgio Valentini and Gianpaolo Carrafiello

J. Imaging 2021, 7(12), 258; https://doi.org/10.3390/jimaging7120258 - 1 Dec 2021

Cited by 3 | Viewed by 3171

Abstract

The aim of this retrospective study is to assess any association between abdominal CT findings and the radiological stage of COVID-19 pneumonia, pulmonary embolism and patient outcomes. We included 158 adult hospitalized COVID-19 patients between 1 March 2020 and 1 March 2021 who [...] Read more.

The aim of this retrospective study is to assess any association between abdominal CT findings and the radiological stage of COVID-19 pneumonia, pulmonary embolism and patient outcomes. We included 158 adult hospitalized COVID-19 patients between 1 March 2020 and 1 March 2021 who underwent 206 abdominal CTs. Two radiologists reviewed all CT images. Pathological findings were classified as acute or not. A subset of patients with inflammatory pathology in ACE2 organs (bowel, biliary tract, pancreas, urinary system) was identified. The radiological stage of COVID pneumonia, pulmonary embolism, overall days of hospitalization, ICU admission and outcome were registered. Univariate statistical analysis coupled with explainable artificial intelligence (AI) techniques were used to discover associations between variables. The most frequent acute findings were bowel abnormalities (n = 58), abdominal fluid (n = 42), hematomas (n = 28) and acute urologic conditions (n = 8). According to univariate statistical analysis, pneumonia stage > 2 was significantly associated with increased frequency of hematomas, active bleeding and fluid-filled colon. The presence of at least one hepatobiliary finding was associated with all the COVID-19 stages > 0. Free abdominal fluid, acute pathologies in ACE2 organs and fluid-filled colon were associated with ICU admission; free fluid also presented poor patient outcomes. Hematomas and active bleeding with at least a progressive stage of COVID pneumonia. The explainable AI techniques find no strong relationship between variables. Full article

(This article belongs to the Special Issue Bio-Medical Multimodal Methods for Diagnosis, Prognosis, and Outcome Prediction)

► Show Figures

Figure 1

12 pages, 6238 KiB

Open AccessArticle

Monocular 3D Body Shape Reconstruction under Clothing

by Claudio Ferrari, Leonardo Casini, Stefano Berretti and Alberto Del Bimbo

J. Imaging 2021, 7(12), 257; https://doi.org/10.3390/jimaging7120257 - 30 Nov 2021

Cited by 5 | Viewed by 4205

Abstract

Estimating the 3D shape of objects from monocular images is a well-established and challenging task in the computer vision field. Further challenges arise when highly deformable objects, such as human faces or bodies, are considered. In this work, we address the problem of [...] Read more.

Estimating the 3D shape of objects from monocular images is a well-established and challenging task in the computer vision field. Further challenges arise when highly deformable objects, such as human faces or bodies, are considered. In this work, we address the problem of estimating the 3D shape of a human body from single images. In particular, we provide a solution to the problem of estimating the shape of the body when the subject is wearing clothes. This is a highly challenging scenario as loose clothes might hide the underlying body shape to a large extent. To this aim, we make use of a parametric 3D body model, the SMPL, whose parameters describe the body pose and shape of the body. Our main intuition is that the shape parameters associated with an individual should not change whether the subject is wearing clothes or not. To improve the shape estimation under clothing, we train a deep convolutional network to regress the shape parameters from a single image of a person. To increase the robustness to clothing, we build our training dataset by associating the shape parameters of a “minimally clothed” person to other samples of the same person wearing looser clothes. Experimental validation shows that our approach can more accurately estimate body shape parameters with respect to state-of-the-art approaches, even in the case of loose clothes. Full article

(This article belongs to the Special Issue 3D Human Understanding)

► Show Figures

Figure 1

Figure 1
Example of the first 3 principal components of shape variations for the SMPL model. Full article ">Figure 2
Phase 1: the proposed model takes as input an image of a person and regresses the SMPL shape parameters <math display="inline"><semantics> <mi>β</mi> </semantics></math>. Phase 2: the trained model is used to extract <math display="inline"><semantics> <mi>β</mi> </semantics></math> parameters from the minimally clothed images. The parameters are assigned to each image of the same individual and the network if fine-tuned. Full article ">Figure 3
Samples of a same subject in the People Snapshot dataset. The “minimally clothed” version (with tight clothes) is shown on the left, and is used to estimate the shape parameters. These parameters are then assigned to the other sequences with looser clothes (right). Full article ">Figure 4
Qualitative examples of body reconstruction from an image of the People Snapshot (left) and an “in the wild” image (right). The two reconstructions are obtained using the HMR method (left) and our approach after Phase 2 (right). Full article ">Figure 5
Qualitative reconstruction examples from “in the wild” images collected from Internet. Full article ">

12 pages, 3956 KiB

Open AccessArticle

Bay Leaf Extract-Based Near-Infrared Fluorescent Probe for Tissue and Cellular Imaging

by Benilde Adriano, Nycol M. Cotto, Neeraj Chauhan, Vinita Karumuru, Meena Jaggi, Subhash C. Chauhan and Murali M. Yallapu

J. Imaging 2021, 7(12), 256; https://doi.org/10.3390/jimaging7120256 - 30 Nov 2021

Viewed by 3635

Abstract

The development of fluorescence dyes for near-infrared (NIR) fluorescence imaging has been a significant interest for deep tissue imaging. Among many imaging fluoroprobes, indocyanine green (ICG) and its analogues have been used in oncology and other medical applications. However, these imaging agents still [...] Read more.

The development of fluorescence dyes for near-infrared (NIR) fluorescence imaging has been a significant interest for deep tissue imaging. Among many imaging fluoroprobes, indocyanine green (ICG) and its analogues have been used in oncology and other medical applications. However, these imaging agents still experience poor imaging capabilities due to low tumor targetability, photostability, and sensitivity in the biological milieu. Thus, developing a biocompatible NIR imaging dye from natural resources holds the potential of facilitating cancer cell/tissue imaging. Chlorophyll (Chl) has been demonstrated to be a potential candidate for imaging purposes due to its natural NIR absorption qualities and its wide availability in plants and green vegetables. Therefore, it was our aim to assess the fluorescence characteristics of twelve dietary leaves as well as the fluorescence of their Chl extractions. Bay leaf extract, a high-fluorescence agent that showed the highest levels of fluorescence, was further evaluated for its tissue contrast and cellular imaging properties. Overall, this study confirms bay-leaf-associated dye as a NIR fluorescence imaging agent that may have important implications for cellular imaging and image-guided cancer surgery. Full article

► Show Figures

Figure 1

17 pages, 45238 KiB

Open AccessArticle

Evaluation of 2D-/3D-Feet-Detection Methods for Semi-Autonomous Powered Wheelchair Navigation

by Cristian Vilar Giménez, Silvia Krug, Faisal Z. Qureshi and Mattias O’Nils

J. Imaging 2021, 7(12), 255; https://doi.org/10.3390/jimaging7120255 - 30 Nov 2021

Cited by 7 | Viewed by 3082

Abstract

Powered wheelchairs have enhanced the mobility and quality of life of people with special needs. The next step in the development of powered wheelchairs is to incorporate sensors and electronic systems for new control applications and capabilities to improve their usability and the [...] Read more.

Powered wheelchairs have enhanced the mobility and quality of life of people with special needs. The next step in the development of powered wheelchairs is to incorporate sensors and electronic systems for new control applications and capabilities to improve their usability and the safety of their operation, such as obstacle avoidance or autonomous driving. However, autonomous powered wheelchairs require safe navigation in different environments and scenarios, making their development complex. In our research, we propose, instead, to develop contactless control for powered wheelchairs where the position of the caregiver is used as a control reference. Hence, we used a depth camera to recognize the caregiver and measure at the same time their relative distance from the powered wheelchair. In this paper, we compared two different approaches for real-time object recognition using a 3DHOG hand-crafted object descriptor based on a 3D extension of the histogram of oriented gradients (HOG) and a convolutional neural network based on YOLOv4-Tiny. To evaluate both approaches, we constructed Miun-Feet—a custom dataset of images of labeled caregiver’s feet in different scenarios, with backgrounds, objects, and lighting conditions. The experimental results showed that the YOLOv4-Tiny approach outperformed 3DHOG in all the analyzed cases. In addition, the results showed that the recognition accuracy was not improved using the depth channel, enabling the use of a monocular RGB camera only instead of a depth camera and reducing the computational cost and heat dissipation limitations. Hence, the paper proposes an additional method to compute the caregiver’s distance and angle from the Powered Wheelchair (PW) using only the RGB data. This work shows that it is feasible to use the location of the caregiver’s feet as a control signal for the control of a powered wheelchair and that it is possible to use a monocular RGB camera to compute their relative positions. Full article

(This article belongs to the Section Computer Vision and Pattern Recognition)

► Show Figures

Figure 1

13 pages, 2216 KiB

Open AccessArticle

Comparison of Different Image Data Augmentation Approaches

by Loris Nanni, Michelangelo Paci, Sheryl Brahnam and Alessandra Lumini

J. Imaging 2021, 7(12), 254; https://doi.org/10.3390/jimaging7120254 - 27 Nov 2021

Cited by 69 | Viewed by 9020

Abstract

Convolutional neural networks (CNNs) have gained prominence in the research literature on image classification over the last decade. One shortcoming of CNNs, however, is their lack of generalizability and tendency to overfit when presented with small training sets. Augmentation directly confronts this problem [...] Read more.

Convolutional neural networks (CNNs) have gained prominence in the research literature on image classification over the last decade. One shortcoming of CNNs, however, is their lack of generalizability and tendency to overfit when presented with small training sets. Augmentation directly confronts this problem by generating new data points providing additional information. In this paper, we investigate the performance of more than ten different sets of data augmentation methods, with two novel approaches proposed here: one based on the discrete wavelet transform and the other on the constant-Q Gabor transform. Pretrained ResNet50 networks are finetuned on each augmentation method. Combinations of these networks are evaluated and compared across four benchmark data sets of images representing diverse problems and collected by instruments that capture information at different scales: a virus data set, a bark data set, a portrait dataset, and a LIGO glitches data set. Experiments demonstrate the superiority of this approach. The best ensemble proposed in this work achieves state-of-the-art (or comparable) performance across all four data sets. This result shows that varying data augmentation is a feasible way for building an ensemble of classifiers for image classification. Full article

(This article belongs to the Special Issue Color Texture Classification)

► Show Figures

Figure 1

13 pages, 1645 KiB

Open AccessReview

Principles and Perspectives of Radiographic Imaging with Muons

by Luigi Cimmino

J. Imaging 2021, 7(12), 253; https://doi.org/10.3390/jimaging7120253 - 26 Nov 2021

Cited by 7 | Viewed by 4398

Abstract

Radiographic imaging with muons, also called Muography, is based on the measurement of the absorption of muons, generated by the interaction of cosmic rays with the earth’s atmosphere, in matter. Muons are elementary particles with high penetrating power, a characteristic that makes them [...] Read more.

Radiographic imaging with muons, also called Muography, is based on the measurement of the absorption of muons, generated by the interaction of cosmic rays with the earth’s atmosphere, in matter. Muons are elementary particles with high penetrating power, a characteristic that makes them capable of crossing bodies of dimensions of the order of hundreds of meters. The interior of bodies the size of a pyramid or a volcano can be seen directly with the use of this technique, which can rely on highly segmented muon trackers. Since the muon flux is distributed in energy over a wide spectrum that depends on the direction of incidence, the main difference with radiography made with X-rays is in the source. The source of muons is not tunable, neither in energy nor in direction; to improve the signal-to-noise ratio, muography requires large instrumentation, long time data acquisition and high background rejection capacity. Here, we present the principles of the Muography, illustrating how radiographic images can be obtained, starting from the measurement of the attenuation of the muon flux through an object. It will then be discussed how recent technologies regarding artificial intelligence can give an impulse to this methodology in order to improve its results. Full article

(This article belongs to the Special Issue X-ray Digital Radiography and Computed Tomography)

► Show Figures

Figure 1

Figure 1
Energy spectra of muons at <math display="inline"><semantics> <msup> <mn>0</mn> <mo>∘</mo> </msup> </semantics></math> (black dots) and at <math display="inline"><semantics> <msup> <mn>75</mn> <mo>∘</mo> </msup> </semantics></math> (white dots) as in [<a href="#B31-jimaging-07-00253" class="html-bibr">31</a>]. Full article ">Figure 2
The angles <math display="inline"><semantics> <mi>θ</mi> </semantics></math> and <math display="inline"><semantics> <msup> <mi>θ</mi> <mo>*</mo> </msup> </semantics></math> formed by an incoming muon, respectively, with the normal axis to the earth surface and with the normal axis to the top of the atmosphere. Full article ">Figure 3
Trends in the ratio between the flux at different altitudes h and the flux at sea level as a function of the muons energy. The comparison of the curves gives the expected percentage change in the flux of muons at different altitudes. Full article ">Figure 4
Schematic of the simulated scenario. Detectors (not in scale) placed in the positions A, B and C. Full article ">Figure 5
A spherical cavity (left) and its 3D reconstruction as seen from the observation point of the detectors C (center) and B (right) as in [<a href="#B48-jimaging-07-00253" class="html-bibr">48</a>]. Full article ">Figure 6
The cube (in gray) predicted by the supervised algorithm compared with the unknown object (left in blue) and with the 3D reconstruction of the unknown object (right in red). Full article ">Figure 7
Denoising process of the 3D reconstructed object. The XY-view on the right shows the predicted object (blue), the 3D reconstructed object (red) and the reference object (gray) in the space constrained by the viewing angle of the detectors placed in positions A, B and C. Full article ">

29 pages, 4655 KiB

Open AccessReview

Roadmap on Digital Holography-Based Quantitative Phase Imaging

by Vinoth Balasubramani, Małgorzata Kujawińska, Cédric Allier, Vijayakumar Anand, Chau-Jern Cheng, Christian Depeursinge, Nathaniel Hai, Saulius Juodkazis, Jeroen Kalkman, Arkadiusz Kuś, Moosung Lee, Pierre J. Magistretti, Pierre Marquet, Soon Hock Ng, Joseph Rosen, Yong Keun Park and Michał Ziemczonok

J. Imaging 2021, 7(12), 252; https://doi.org/10.3390/jimaging7120252 - 26 Nov 2021

Cited by 49 | Viewed by 8422

Abstract

Quantitative Phase Imaging (QPI) provides unique means for the imaging of biological or technical microstructures, merging beneficial features identified with microscopy, interferometry, holography, and numerical computations. This roadmap article reviews several digital holography-based QPI approaches developed by prominent research groups. It also briefly [...] Read more.

Quantitative Phase Imaging (QPI) provides unique means for the imaging of biological or technical microstructures, merging beneficial features identified with microscopy, interferometry, holography, and numerical computations. This roadmap article reviews several digital holography-based QPI approaches developed by prominent research groups. It also briefly discusses the present and future perspectives of 2D and 3D QPI research based on digital holographic microscopy, holographic tomography, and their applications. Full article

(This article belongs to the Special Issue Digital Holography: Development and Application)

► Show Figures

Figure 1

20 pages, 12131 KiB

Open AccessArticle

Formation of Fused Images of the Land Surface from Radar and Optical Images in Spatially Distributed On-Board Operational Monitoring Systems

by Vadim A. Nenashev and Igor G. Khanykov

J. Imaging 2021, 7(12), 251; https://doi.org/10.3390/jimaging7120251 - 25 Nov 2021

Cited by 12 | Viewed by 2425

Abstract

This paper considers the issues of image fusion in a spatially distributed small-size on-board location system for operational monitoring. The purpose of this research is to develop a new method for the formation of fused images of the land surface based on data [...] Read more.

This paper considers the issues of image fusion in a spatially distributed small-size on-board location system for operational monitoring. The purpose of this research is to develop a new method for the formation of fused images of the land surface based on data obtained from optical and radar devices operated from two-position spatially distributed systems of small aircraft, including unmanned aerial vehicles. The advantages of the method for integrating information from radar and optical information-measuring systems are justified. The combined approach allows removing the limitations of each separate system. The practicality of choosing the integration of information from several widely used variants of heterogeneous sources is shown. An iterative approach is used in the method for combining multi-angle location images. This approach improves the quality of synthesis and increases the accuracy of integration, as well as improves the information content and reliability of the final fused image by using the pixel clustering algorithm, which produces many partitions into clusters. The search for reference points on isolated contours is carried out on a pair of left and right images of the docked image from the selected partition. For these reference points, a functional transformation is determined. Having applied it to the original multi-angle heterogeneous images, the degree of correlation of the fused image is assessed. Both the position of the reference points of the contour and the desired functional transformation itself are refined until the quality assessment of the fusion becomes acceptable. The type of functional transformation is selected based on clustered images and then applied to the original multi-angle heterogeneous images. This process is repeated for clustered images with greater granularity in case if quality assessment of the fusion is considered to be poor. At each iteration, there is a search for pairs of points of the contour of the isolated areas. Areas are isolated with the use of two image segmentation methods. Experiments on the formation of fused images are presented. The result of the research is the proposed method for integrating information obtained from a two-position airborne small-sized radar system and an optical location system. The implemented method can improve the information content, quality, and reliability of the finally established fused image of the land surface. Full article

► Show Figures

Figure 1

Journal Menu

Journal Browser

J. Imaging, Volume 7, Issue 12 (December 2021) – 33 articles

Further Information

Guidelines

MDPI Initiatives

Follow MDPI