Entropy

13 pages, 4382 KiB

Open AccessEditor’s ChoiceArticle

Eigenvalues of Two-State Quantum Walks Induced by the Hadamard Walk

by Shimpei Endo, Takako Endo, Takashi Komatsu and Norio Konno

Entropy 2020, 22(1), 127; https://doi.org/10.3390/e22010127 - 20 Jan 2020

Cited by 7 | Viewed by 4316

Existence of the eigenvalues of the discrete-time quantum walks is deeply related to localization of the walks. We revealed, for the first time, the distributions of the eigenvalues given by the splitted generating function method (the SGF method) of the space-inhomogeneous quantum walks [...] Read more.

Existence of the eigenvalues of the discrete-time quantum walks is deeply related to localization of the walks. We revealed, for the first time, the distributions of the eigenvalues given by the splitted generating function method (the SGF method) of the space-inhomogeneous quantum walks in one dimension we had treated in our previous studies. Especially, we clarified the characteristic parameter dependence for the distributions of the eigenvalues with the aid of numerical simulation. Full article

(This article belongs to the Special Issue Quantum Walks and Related Issues)

► Show Figures

Figure 1

14 pages, 636 KiB

Open AccessArticle

A Standardized Project Gutenberg Corpus for Statistical Analysis of Natural Language and Quantitative Linguistics

by Martin Gerlach and Francesc Font-Clos

Entropy 2020, 22(1), 126; https://doi.org/10.3390/e22010126 - 20 Jan 2020

Cited by 43 | Viewed by 9453

Abstract

The use of Project Gutenberg (PG) as a text corpus has been extremely popular in statistical analysis of language for more than 25 years. However, in contrast to other major linguistic datasets of similar importance, no consensual full version of PG exists to [...] Read more.

The use of Project Gutenberg (PG) as a text corpus has been extremely popular in statistical analysis of language for more than 25 years. However, in contrast to other major linguistic datasets of similar importance, no consensual full version of PG exists to date. In fact, most PG studies so far either consider only a small number of manually selected books, leading to potential biased subsets, or employ vastly different pre-processing strategies (often specified in insufficient details), raising concerns regarding the reproducibility of published results. In order to address these shortcomings, here we present the Standardized Project Gutenberg Corpus (SPGC), an open science approach to a curated version of the complete PG data containing more than 50,000 books and more than

3 \times 10^{9}

word-tokens. Using different sources of annotated metadata, we not only provide a broad characterization of the content of PG, but also show different examples highlighting the potential of SPGC for investigating language variability across time, subjects, and authors. We publish our methodology in detail, the code to download and process the data, as well as the obtained corpus itself on three different levels of granularity (raw text, timeseries of word tokens, and counts of words). In this way, we provide a reproducible, pre-processed, full-size version of Project Gutenberg as a new scientific resource for corpus linguistics, natural language processing, and information retrieval. Full article

(This article belongs to the Special Issue Information Theory and Language)

► Show Figures

Figure 1

Figure 1
Sketch of the pre-processing pipeline of the Project Gutenberg (PG) data. The folder structure (left) organizes each PG book on four different levels of granularity, see example books (middle): raw, text, tokens, and counts. On the right we show the basic python commands used in the pre-processing. Full article ">Figure 2
Basic summary statistics from the processed PG data. (a) Number of books with a text length larger than m; (b) Number of books which are compatible with being published in year t, i.e., year of author’s birth is 20 years prior and year of author’s death is after t; (c) Number of books (left axis) and number of tokens (right axis) which are assigned to a given language based on the metadata. en: English, fr: French, fi: Finnish, de: German, nl: Dutch, it: Italian, es: Spanish, pt: Portuguese, zh: Chinese, el: Greek, sv: Swedish, hu: Hungarian, eo: Esperanto, la: Latin, da: Danish, tl: Tagalog, ca: Catalan, pl: Polish, ja: Japanese, no: Norwegian, cy: Welsh, cs: Czech. Full article ">Figure 3
Comparison between bookshelf labels (top, green) and subject labels (bottom, red). (a,c) Number of labels with a given number of books; (b,d) Fraction of books with a given number of labels. Full article ">Figure 4
2-dimensional embedding shows clustering of books from the same bookshelf. Approximate visualization of the pair-wise distances between 370 PG books using Uniform Manifold Approximation and Projection (UMAP) (for details, see <a href="#sec4-entropy-22-00126" class="html-sec">Section 4</a>). Each dot corresponds to one book colored according to the bookshelf membership. Full article ">Figure 5
Distance between books from the same author is significantly smaller than distance between books from different authors. For each author, the boxplots shows the <math display="inline"><semantics> <mrow> <mn>5</mn> <mo>,</mo> <mn>25</mn> <mo>,</mo> <mn>50</mn> <mo>,</mo> <mn>75</mn> <mo>,</mo> <mn>95</mn> </mrow> </semantics></math>-percentile of the distribution of distances from <math display="inline"><semantics> <mrow> <mn>1000</mn> </mrow> </semantics></math> pairs of books from the same author (green) and to a different author (gray). Full article ">Figure 6
Distance between books increases with their time separation. Average and standard error of the distance between 1000 pairs of books, where the two books in each pair is drawn from two different 20-year time intervals. We fix the first interval and continuously increase the second time interval. Full article ">

19 pages, 2160 KiB

Open AccessArticle

Low-Complexity Rate-Distortion Optimization of Sampling Rate and Bit-Depth for Compressed Sensing of Images

by Qunlin Chen, Derong Chen, Jiulu Gong and Jie Ruan

Entropy 2020, 22(1), 125; https://doi.org/10.3390/e22010125 - 20 Jan 2020

Cited by 5 | Viewed by 3432

Abstract

Compressed sensing (CS) offers a framework for image acquisition, which has excellent potential in image sampling and compression applications due to the sub-Nyquist sampling rate and low complexity. In engineering practices, the resulting CS samples are quantized by finite bits for transmission. In [...] Read more.

Compressed sensing (CS) offers a framework for image acquisition, which has excellent potential in image sampling and compression applications due to the sub-Nyquist sampling rate and low complexity. In engineering practices, the resulting CS samples are quantized by finite bits for transmission. In circumstances where the bit budget for image transmission is constrained, knowing how to choose the sampling rate and the number of bits per measurement (bit-depth) is essential for the quality of CS reconstruction. In this paper, we first present a bit-rate model that considers the compression performance of CS, quantification, and entropy coder. The bit-rate model reveals the relationship between bit rate, sampling rate, and bit-depth. Then, we propose a relative peak signal-to-noise ratio (PSNR) model for evaluating distortion, which reveals the relationship between relative PSNR, sampling rate, and bit-depth. Finally, the optimal sampling rate and bit-depth are determined based on the rate-distortion (RD) criteria with the bit-rate model and the relative PSNR model. The experimental results show that the actual bit rate obtained by the optimized sampling rate and bit-depth is very close to the target bit rate. Compared with the traditional CS coding method with a fixed sampling rate, the proposed method provides better rate-distortion performance, and the additional calculation amount amounts to less than 1%. Full article

(This article belongs to the Section Signal and Data Analysis)

► Show Figures

Figure 1

4 pages, 1717 KiB

Open AccessEditorial

Entropy 2019 Best Paper Award

by Entropy Editorial Office

Entropy 2020, 22(1), 124; https://doi.org/10.3390/e22010124 - 20 Jan 2020

Viewed by 5280

Abstract

On behalf of the Editor-in-Chief, Prof [...] Full article

12 pages, 2083 KiB

Open AccessArticle

Analyzing Uncertainty in Complex Socio-Ecological Networks

by Ana D. Maldonado, María Morales, Pedro A. Aguilera and Antonio Salmerón

Entropy 2020, 22(1), 123; https://doi.org/10.3390/e22010123 - 19 Jan 2020

Cited by 12 | Viewed by 3622

Abstract

Socio-ecological systems are recognized as complex adaptive systems whose multiple interactions might change as a response to external or internal changes. Due to its complexity, the behavior of the system is often uncertain. Bayesian networks provide a sound approach for handling complex domains [...] Read more.

Socio-ecological systems are recognized as complex adaptive systems whose multiple interactions might change as a response to external or internal changes. Due to its complexity, the behavior of the system is often uncertain. Bayesian networks provide a sound approach for handling complex domains endowed with uncertainty. The aim of this paper is to analyze the impact of the Bayesian network structure on the uncertainty of the model, expressed as the Shannon entropy. In particular, three strategies for model structure have been followed: naive Bayes (NB), tree augmented network (TAN) and network with unrestricted structure (GSS). Using these network structures, two experiments are carried out: (1) the impact of the Bayesian network structure on the entropy of the model is assessed and (2) the entropy of the posterior distribution of the class variable obtained from the different structures is compared. The results show that GSS constantly outperforms both NB and TAN when it comes to evaluating the uncertainty of the entire model. On the other hand, NB and TAN yielded lower entropy values of the posterior distribution of the class variable, which makes them preferable when the goal is to carry out predictions. Full article

(This article belongs to the Special Issue Computation in Complex Networks)

► Show Figures

Figure 1

15 pages, 826 KiB

Open AccessArticle

Stabilization of Port Hamiltonian Chaotic Systems with Hidden Attractors by Adaptive Terminal Sliding Mode Control

by Ahmad Taher Azar and Fernando E. Serrano

Entropy 2020, 22(1), 122; https://doi.org/10.3390/e22010122 - 19 Jan 2020

Cited by 25 | Viewed by 3690

Abstract

In this study, the design of an adaptive terminal sliding mode controller for the stabilization of port Hamiltonian chaotic systems with hidden attractors is proposed. This study begins with the design methodology of a chaotic oscillator with a hidden attractor implementing the topological [...] Read more.

In this study, the design of an adaptive terminal sliding mode controller for the stabilization of port Hamiltonian chaotic systems with hidden attractors is proposed. This study begins with the design methodology of a chaotic oscillator with a hidden attractor implementing the topological framework for its respective design. With this technique it is possible to design a 2-D chaotic oscillator, which is then converted into port-Hamiltonia to track and analyze these models for the stabilization of the hidden chaotic attractors created by this analysis. Adaptive terminal sliding mode controllers (ATSMC) are built when a Hamiltonian system has a chaotic behavior and a hidden attractor is detected. A Lyapunov approach is used to formulate the adaptive device controller by creating a control law and the adaptive law, which are used online to make the system states stable while at the same time suppressing its chaotic behavior. The empirical tests obtaining the discussion and conclusions of this thesis should verify the theoretical findings. Full article

(This article belongs to the Special Issue Nonlinear Dynamics and Entropy of Complex Systems with Hidden and Self-Excited Attractors II)

► Show Figures

Figure 1

27 pages, 1087 KiB

Open AccessArticle

Linear Programming and Fuzzy Optimization to Substantiate Investment Decisions in Tangible Assets

by Marcel-Ioan Boloș, Ioana-Alexandra Bradea and Camelia Delcea

Entropy 2020, 22(1), 121; https://doi.org/10.3390/e22010121 - 19 Jan 2020

Cited by 4 | Viewed by 4338

Abstract

This paper studies the problem of tangible assets acquisition within the company by proposing a new hybrid model that uses linear programming and fuzzy numbers. Regarding linear programming, two methods were implemented in the model, namely: the graphical method and the primal simplex [...] Read more.

This paper studies the problem of tangible assets acquisition within the company by proposing a new hybrid model that uses linear programming and fuzzy numbers. Regarding linear programming, two methods were implemented in the model, namely: the graphical method and the primal simplex algorithm. This hybrid model is proposed for solving investment decision problems, based on decision variables, objective function coefficients, and a matrix of constraints, all of them presented in the form of triangular fuzzy numbers. Solving the primal simplex algorithm using fuzzy numbers and coefficients, allowed the results of the linear programming problem to also be in the form of fuzzy variables. The fuzzy variables compared to the crisp variables allow the determination of optimal intervals for which the objective function has values depending on the fuzzy variables. The major advantage of this model is that the results are presented as value ranges that intervene in the decision-making process. Thus, the company’s decision makers can select any of the result values as they satisfy two basic requirements namely: minimizing/maximizing the objective function and satisfying the basic requirements regarding the constraints resulting from the company’s activity. The paper is accompanied by a practical example. Full article

(This article belongs to the Special Issue Information Theories Based on Belief Functions for Decision-Making Support)

► Show Figures

Figure 1

12 pages, 385 KiB

Open AccessArticle

Generalized Independence in the q-Voter Model: How Do Parameters Influence the Phase Transition?

by Angelika Abramiuk and Katarzyna Sznajd-Weron

Entropy 2020, 22(1), 120; https://doi.org/10.3390/e22010120 - 19 Jan 2020

Cited by 10 | Viewed by 3259

Abstract

We study the q-voter model with flexibility, which allows for describing a broad spectrum of independence from zealots, inflexibility, or stubbornness through noisy voters to self-anticonformity. Analyzing the model within the pair approximation allows us to derive the analytical formula for the [...] Read more.

We study the q-voter model with flexibility, which allows for describing a broad spectrum of independence from zealots, inflexibility, or stubbornness through noisy voters to self-anticonformity. Analyzing the model within the pair approximation allows us to derive the analytical formula for the critical point, below which an ordered (agreement) phase is stable. We determine the role of flexibility, which can be understood as an amount of variability associated with an independent behavior, as well as the role of the average network degree in shaping the character of the phase transition. We check the existence of the scaling relation, which previously was derived for the Sznajd model. We show that the scaling is universal, in a sense that it does not depend neither on the size of the group of influence nor on the average network degree. Analyzing the model in terms of the rescaled parameter, we determine the critical point, the jump of the order parameter, as well as the width of the hysteresis as a function of the average network degree

⟨ k ⟩

and the size of the group of influence q. Full article

(This article belongs to the Special Issue Statistical Mechanics of Complex Systems)

► Show Figures

Figure 1

20 pages, 1055 KiB

Open AccessArticle

Activeness and Loyalty Analysis in Event-Based Social Networks

by Thanh Trinh, Dingming Wu, Joshua Zhexue Huang and Muhammad Azhar

Entropy 2020, 22(1), 119; https://doi.org/10.3390/e22010119 - 18 Jan 2020

Cited by 11 | Viewed by 3866

Abstract

Event-based social networks (EBSNs) are widely used to create online social groups and organize offline events for users. Activeness and loyalty are crucial characteristics of these online social groups in terms of determining the growth or inactiveness of the social groups in a [...] Read more.

Event-based social networks (EBSNs) are widely used to create online social groups and organize offline events for users. Activeness and loyalty are crucial characteristics of these online social groups in terms of determining the growth or inactiveness of the social groups in a specific time frame. However, there is less research on these concepts to clarify the existence of groups in event-based social networks. In this paper, we study the problem of group activeness and user loyalty to provide a novel insight into online social networks. First, we analyze the structure of EBSNs and generate features from the crawled datasets. Second, we define the concepts of group activeness and user loyalty based on a series of time windows, and propose a method to measure the group activeness. In this proposed method, we first compute a ratio of a number of events between two consecutive time windows. We then develop an association matrix to assign the activeness label for each group after several consecutive time windows. Similarly, we measure the user loyalty in terms of attended events gathered in time windows and treat loyalty as a contributive feature of the group activeness. Finally, three well-known machine learning techniques are used to verify the activeness label and to generate features for each group. As a consequence, we also find a small group of features that are highly correlated and result in higher accuracy as compared to the whole features. Full article

(This article belongs to the Special Issue Social Networks and Information Diffusion II)

► Show Figures

Figure 1

22 pages, 11340 KiB

Open AccessArticle

Entropy-Based Image Fusion with Joint Sparse Representation and Rolling Guidance Filter

by Yudan Liu, Xiaomin Yang, Rongzhu Zhang, Marcelo Keese Albertini, Turgay Celik and Gwanggil Jeon

Entropy 2020, 22(1), 118; https://doi.org/10.3390/e22010118 - 18 Jan 2020

Cited by 22 | Viewed by 5104

Abstract

Image fusion is a very practical technology that can be applied in many fields, such as medicine, remote sensing and surveillance. An image fusion method using multi-scale decomposition and joint sparse representation is introduced in this paper. First, joint sparse representation is applied [...] Read more.

Image fusion is a very practical technology that can be applied in many fields, such as medicine, remote sensing and surveillance. An image fusion method using multi-scale decomposition and joint sparse representation is introduced in this paper. First, joint sparse representation is applied to decompose two source images into a common image and two innovation images. Second, two initial weight maps are generated by filtering the two source images separately. Final weight maps are obtained by joint bilateral filtering according to the initial weight maps. Then, the multi-scale decomposition of the innovation images is performed through the rolling guide filter. Finally, the final weight maps are used to generate the fused innovation image. The fused innovation image and the common image are combined to generate the ultimate fused image. The experimental results show that our method’s average metrics are: mutual information (

M I

)—5.3377, feature mutual information (

F M I

)—0.5600, normalized weighted edge preservation value (

Q^{A B / F}

)—0.6978 and nonlinear correlation information entropy (

N C I E

)—0.8226. Our method can achieve better performance compared to the state-of-the-art methods in visual perception and objective quantification. Full article

(This article belongs to the Special Issue Entropy-Based Algorithms for Signal Processing)

► Show Figures

Figure 1

15 pages, 2398 KiB

Open AccessArticle

Chemical Reaction Networks Possess Intrinsic, Temperature-Dependent Functionality

by Stephan O. Adler and Edda Klipp

Entropy 2020, 22(1), 117; https://doi.org/10.3390/e22010117 - 18 Jan 2020

Cited by 6 | Viewed by 4322

Abstract

Temperature influences the life of many organisms in various ways. A great number of organisms live under conditions where their ability to adapt to changes in temperature can be vital and largely determines their fitness. Understanding the mechanisms and principles underlying this ability [...] Read more.

Temperature influences the life of many organisms in various ways. A great number of organisms live under conditions where their ability to adapt to changes in temperature can be vital and largely determines their fitness. Understanding the mechanisms and principles underlying this ability to adapt can be of great advantage, for example, to improve growth conditions for crops and increase their yield. In times of imminent, increasing climate change, this becomes even more important in order to find strategies and help crops cope with these fundamental changes. There is intense research in the field of acclimation that comprises fluctuations of various environmental conditions, but most acclimation research focuses on regulatory effects and the observation of gene expression changes within the examined organism. As thermodynamic effects are a direct consequence of temperature changes, these should necessarily be considered in this field of research but are often neglected. Additionally, compensated effects might be missed even though they are equally important for the organism, since they do not cause observable changes, but rather counteract them. In this work, using a systems biology approach, we demonstrate that even simple network motifs can exhibit temperature-dependent functional features resulting from the interplay of network structure and the distribution of activation energies over the involved reactions. The demonstrated functional features are (i) the reversal of fluxes within a linear pathway, (ii) a thermo-selective branched pathway with different flux modes and (iii) the increased flux towards carbohydrates in a minimal Calvin cycle that was designed to demonstrate temperature compensation within reaction networks. Comparing a system’s response to either temperature changes or changes in enzyme activity we also dissect the influence of thermodynamic changes versus genetic regulation. By this, we expand the scope of thermodynamic modelling of biochemical processes by addressing further possibilities and effects, following established mathematical descriptions of biophysical properties. Full article

(This article belongs to the Special Issue Information Flow and Entropy Production in Biomolecular Networks)

► Show Figures

Figure 1

Figure 1
Energy diagram for reversible Michaelis-Menten kinetics. The figure sketches the different levels of Gibbs free energy G1 to G5 for a reversible enzymatic reaction with a substrate S, an enzyme E and a product P. The relation of these energy levels is a determining factor for the overall reaction rate and direction. Full article ">Figure 2
Illustration of exponential temperature response curves. Panel (A) shows a typical exponential temperature response curve of a reaction flux, following the Arrhenius equation. Panel (B) illustrates a case where the forward flux exceeds the backward flux of a reversible reaction at temperature TJ=0. The resulting net flux at this temperature is zero and changes its direction when passing it. Full article ">Figure 3
Flux reversal in a linear pathway. The upper part of (A) shows the structure of the reversible linear pathway, comprising the two species A and B. The rate constants of the forward reactions are indicated by k1 to k3, while k−1 to k−3 are the rate constants of the backward reactions. The lower part of (A) shows the corresponding net fluxes for a temperature range from 273 to 308 K. The three panels of (B) illustrate the system’s response to continuous cyclical temperature changes. The upper panel shows the time courses for three different temperatures oscillating by ± 5 K around 290 K (blue), 300 K (green) and 310 K (orange). The middle panel shows the resulting temporal changes in concentrations of species A and B. The flux changes are depicted in the lower panel. The colors of the concentration and flux curves match the colors of their corresponding temperature curves. Full article ">Figure 4
Energy diagram for a single reaction within a linear, reversible pathway. Panel (A) shows the Gibbs free energies of activation for the forward (ΔG+i) and backward (ΔG−i) direction of reaction i of the linear, reversible pathway shown in panel (B). Here, ΔGi is the difference in Gibbs free energy of the substrate Si−1 and the product Si. Full article ">Figure 5
Thermo-selective, branched pathway. The figure shows the structure of a reversible branched pathway with three species (upper part) and the net fluxes (considering forward and backward direction) for all involved reactions (lower part). The numbers in top left corner in each of the panels denote the corresponding reaction. Here we see three specifiable temperature regions with distinctive flux directions (colored areas). While the flux comes in from the upper branch and flows out through the lower branch within the blue temperature region, it can change to flowing outwards at both branches (red region) and eventually invert the initial flux direction with rising temperature (green region). Full article ">Figure 6
Comparison of the effect of gene expression changes versus temperature changes. Considering a minimal branched network, we changed either the concentrations of the three enzymes individually or the temperature that affects all three reactions at the same time. The panels in the top row show the flux changes in response to changes in temperature (left) and enzyme concentration (right) separately. Notice the reversal of flux direction of reaction 1 (blue). The bottom row shows three heatmaps illustrating the combined effects of enzyme 2 and temperature T on the concentration of species S (left). The relative change of S compared to the case that only enzyme 2 is varied and T is kept at 290 K (middle), and the relative change of S compared to the case that temperature is varied and enzyme 2 is kept at concentration 1. Full article ">Figure 7
Increased flux towards carbohydrates in a minimal Calvin cycle. The figure shows the structure of a Calvin cycle model designed by P. Ruoff et al. (2007) [<a href="#B10-entropy-22-00117" class="html-bibr">10</a>] which comprises only its most essential processes as lumped single reactions. The structure of this network has the potential to exhibit an increased flux towards carbohydrates at lower temperatures that is based on the distribution of activation energies and pre-exponential factors of the reactions within the network. An accumulation of carbohydrates at lower temperatures has been observed and reported frequently. Full article ">Figure 8
Temperature response of σ for a linear reversible reaction system. The figure shows the temperature response of the entropy production density σ of all internal reactions and reaction 2 from the linear pathway illustrated in <a href="#entropy-22-00117-f003" class="html-fig">Figure 3</a>, separately. The entropy production vanishes at TJ=0, the temperature at which the net flux is zero. It increases for both higher and lower temperatures. Full article ">

20 pages, 6819 KiB

Open AccessArticle

Permutation Entropy and Statistical Complexity in Mild Cognitive Impairment and Alzheimer’s Disease: An Analysis Based on Frequency Bands

by Ignacio Echegoyen, David López-Sanz, Johann H. Martínez, Fernando Maestú and Javier M. Buldú

Entropy 2020, 22(1), 116; https://doi.org/10.3390/e22010116 - 18 Jan 2020

Cited by 21 | Viewed by 4580

Abstract

We present one of the first applications of Permutation Entropy (PE) and Statistical Complexity (SC) (measured as the product of PE and Jensen-Shanon Divergence) on Magnetoencephalography (MEG) recordings of 46 subjects suffering from Mild Cognitive Impairment (MCI), 17 individuals diagnosed with Alzheimer’s Disease [...] Read more.

We present one of the first applications of Permutation Entropy (PE) and Statistical Complexity (SC) (measured as the product of PE and Jensen-Shanon Divergence) on Magnetoencephalography (MEG) recordings of 46 subjects suffering from Mild Cognitive Impairment (MCI), 17 individuals diagnosed with Alzheimer’s Disease (AD) and 48 healthy controls. We studied the differences in PE and SC in broadband signals and their decomposition into frequency bands (

δ

,

θ

,

α

and

β

), considering two modalities: (i) raw time series obtained from the magnetometers and (ii) a reconstruction into cortical sources or regions of interest (ROIs). We conducted our analyses at three levels: (i) at the group level we compared SC in each frequency band and modality between groups; (ii) at the individual level we compared how the [PE, SC] plane differs in each modality; and (iii) at the local level we explored differences in scalp and cortical space. We recovered classical results that considered only broadband signals and found a nontrivial pattern of alterations in each frequency band, showing that SC does not necessarily decrease in AD or MCI. Full article

(This article belongs to the Special Issue Permutation Entropy: Theory and Applications)

► Show Figures

Figure 1

17 pages, 1176 KiB

Open AccessArticle

Association between Mean Heart Rate and Recurrence Quantification Analysis of Heart Rate Variability in End-Stage Renal Disease

by Martín Calderón-Juárez, Gertrudis Hortensia González-Gómez, Juan C. Echeverría, Héctor Pérez-Grovas and Claudia Lerma

Entropy 2020, 22(1), 114; https://doi.org/10.3390/e22010114 - 18 Jan 2020

Cited by 11 | Viewed by 4424

Abstract

Linear heart rate variability (HRV) indices are dependent on the mean heart rate, which has been demonstrated in different models (from sinoatrial cells to humans). The association between nonlinear HRV indices, including those provided by recurrence plot quantitative analysis (RQA), and the mean [...] Read more.

Linear heart rate variability (HRV) indices are dependent on the mean heart rate, which has been demonstrated in different models (from sinoatrial cells to humans). The association between nonlinear HRV indices, including those provided by recurrence plot quantitative analysis (RQA), and the mean heart rate (or the mean cardiac period, also called meanNN) has been scarcely studied. For this purpose, we analyzed RQA indices of five minute-long HRV time series obtained in the supine position and during active standing from 30 healthy subjects and 29 end-stage renal disease (ESRD) patients (before and after hemodialysis). In the supine position, ESRD patients showed shorter meanNN (i.e., faster heart rate) and decreased variability compared to healthy subjects. The healthy subjects responded to active standing by shortening the meanNN and decreasing HRV indices to reach similar values of ESRD patients. Bivariate correlations between all RQA indices and meanNN were significant in healthy subjects and ESRD after hemodialysis and for most RQA indices in ESRD patients before hemodialysis. Multiple linear regression analyses showed that RQA indices were also dependent on the position and the ESRD condition. Then, future studies should consider the association among RQA indices, meanNN, and these other factors for a correct interpretation of HRV. Full article

► Show Figures

Figure 1

Figure 1
Assessment of the autocorrelation function (upper panels) and false nearest neighbors method (lower panels) for each recording (thin lines) and averaged values per group (thick lines). Black lines correspond to healthy subjects, green lines correspond to ESRD patients before hemodialysis, and magenta lines correspond to ESRD patients after hemodialysis. Full article ">Figure 2
Examples of RR time series and recurrence plots from a healthy subject (upper panels) and an end-stage renal disease (ESRD) patient (lower panels). The recording from the healthy subject shows a reduction in RR intervals and their variability, as a consequence of changing to an active standing position. The recurrence plot shows extended global diagonal lines formed by short diagonals covering a regularly spaced pattern of the plot surface. In contrast, during the standing position, there is an increase in white zones (i.e., there are less accessible dynamic states in the central area). In the patient before hemodialysis, there is a marked reduction in the RR intervals and their associated variability, even in the supine position. Its recurrence plot quantitative analysis (RQA) plot has less regularly extended diagonals with more concentration around the identity line and its parallel lines. During the active standing condition, the plot is almost empty; there are very fewer accessible dynamical states. In the same patient after hemodialysis, there is a slight improvement in the RR intervals during the supine condition compared to those before hemodialysis, which is also reflected in the RQA plot. Again, during the active standing challenge, there is a reduction in RR intervals and reachable dynamical states. Full article ">

19 pages, 4838 KiB

Open AccessEditor’s ChoiceArticle

Energy and Exergy Evaluation of a Two-Stage Axial Vapour Compressor on the LNG Carrier

by Igor Poljak, Josip Orović, Vedran Mrzljak and Dean Bernečić

Entropy 2020, 22(1), 115; https://doi.org/10.3390/e22010115 - 17 Jan 2020

Cited by 7 | Viewed by 4971

Abstract

Data from a two-stage axial vapor cryogenic compressor on the dual-fuel diesel–electric (DFDE) liquefied natural gas (LNG) carrier were measured and analyzed to investigate compressor energy and exergy efficiency in real exploitation conditions. The running parameters of the two-stage compressor were collected while [...] Read more.

Data from a two-stage axial vapor cryogenic compressor on the dual-fuel diesel–electric (DFDE) liquefied natural gas (LNG) carrier were measured and analyzed to investigate compressor energy and exergy efficiency in real exploitation conditions. The running parameters of the two-stage compressor were collected while changing the main propeller shafts rpm. As the compressor supply of vaporized gas to the main engines increases, so does the load and rpm in propulsion electric motors, and vice versa. The results show that when the main engine load varied from 46 to 56 rpm at main propulsion shafts increased mass flow rate of vaporized LNG at a two-stage compressor has an influence on compressor performance. Compressor average energy efficiency is around 50%, while the exergy efficiency of the compressor is significantly lower in all measured ranges and on average is around 34%. The change in the ambient temperature from 0 to 50 °C also influences the compressor’s exergy efficiency. Higher exergy efficiency is achieved at lower ambient temperatures. As temperature increases, overall compressor exergy efficiency decreases by about 7% on average over the whole analyzed range. The proposed new concept of energy-saving and increasing the compressor efficiency based on pre-cooling of the compressor second stage is also analyzed. The temperature at the second stage was varied in the range from 0 to −50 °C, which results in power savings up to 26 kW for optimal running regimes. Full article

(This article belongs to the Special Issue Carnot Cycle and Heat Engine Fundamentals and Applications)

► Show Figures

Figure 1

17 pages, 7722 KiB

Open AccessArticle

Entropy-Based Effect Evaluation of Delineators in Tunnels on Drivers’ Gaze Behavior

by Xueyan Han, Yang Shao, Shaowei Yang and Peng Yu

Entropy 2020, 22(1), 113; https://doi.org/10.3390/e22010113 - 17 Jan 2020

Cited by 18 | Viewed by 3647

Abstract

Driving safety in tunnels has always been an issue of great concern. Establishing delineators to improve drivers’ instantaneous cognition of the surrounding environment in tunnels can effectively enhance driver safety. Through a simulation study, this paper explored how delineators affect drivers’ gaze behavior [...] Read more.

Driving safety in tunnels has always been an issue of great concern. Establishing delineators to improve drivers’ instantaneous cognition of the surrounding environment in tunnels can effectively enhance driver safety. Through a simulation study, this paper explored how delineators affect drivers’ gaze behavior (including fixation and scanpath) in tunnels. In addition to analyzing typical parameters, such as fixation position and fixation duration in areas of interest (AOIs), by modeling drivers’ switching process as Markov chains and calculating Shannon’s entropy of the fit Markov model, this paper quantified the complexity of individual switching patterns between AOIs under different delineator configurations and with different road alignments. A total of 25 subjects participated in this research. The results show that setting delineators in tunnels can attract drivers’ attention and make them focus on the pavement. When driving in tunnels equipped with delineators, especially tunnels with both wall delineators and pavement delineators, the participants exhibited a smaller transition entropy

H_{t}

and stationary entropy

H_{s}

, which can greatly reduce drivers’ visual fatigue. Compared with left curve and right curve, participants obtained higher

H_{t}

and

H_{s}

values in the straight section. Full article

(This article belongs to the Section Multidisciplinary Applications)

► Show Figures

Figure 1

Figure 1
Test route alignment design (Qinling Mountain No. 1, No. 2 and No. 3 tunnels of the G5 Expressway). Full article ">Figure 2
The location of delineators in tunnel. Full article ">Figure 3
Simulation view in the simulator. This picture is the simulation shown on the middle liquid crystal display when participants drove in scenario A. Full article ">Figure 4
(a) Straight; (b) Left curve; (c) Right curve. Participants’ fixation positions in three different scenarios (In scenario A, both the wall delineators and pavement delineators were set. In scenario B, only the pavement delineators were set. In scenario C, there were no delineators.) combined with three tunnel alignments (straight, left curve and right curve). Full article ">Figure 5
Participant’s visual area division. Full article ">Figure 6
(a) straight line in scenario A; (b) straight line in scenario B; (c) straight line in scenario C; (d) left curve in scenario A; (e) left curve in scenario B; (f) left curve in scenario C; (g) right curve in scenario A; (h) right curve in scenario B; (i) right curve in scenario C. The percentage of fixation duration in three different scenarios (In scenario A, both the wall delineators and pavement delineators were set. In scenario B, only the pavement delineators were set. In scenario C, there was no delineator.) combined with three tunnel alignments (straight, left curve and right curve) for the 21 participants. There are 9 subgraphs, in each subgraph, there are 21 concentric circles. Each concentric circle represents one participant. The six different colors in each concentric circle represent the fixation duration percentage of participant for six different areas, the black, red, blue, green, purple, and yellow parts represent the proportion of gaze time spent in the pavement area (PA), the central area (CA), the top wall (TW), the left wall (LW), the right wall (RW) and the white space (WS), respectively. Full article ">Figure 7
(a) <math display="inline"><semantics> <msub> <mi>H</mi> <mi>t</mi> </msub> </semantics></math> in straight line; (b) <math display="inline"><semantics> <msub> <mi>H</mi> <mi>s</mi> </msub> </semantics></math> in straight line; (c) <math display="inline"><semantics> <msub> <mi>H</mi> <mi>t</mi> </msub> </semantics></math> on left curve; (d) <math display="inline"><semantics> <msub> <mi>H</mi> <mi>s</mi> </msub> </semantics></math> on left curve; (e) <math display="inline"><semantics> <msub> <mi>H</mi> <mi>t</mi> </msub> </semantics></math> on right curve; (f) <math display="inline"><semantics> <msub> <mi>H</mi> <mi>s</mi> </msub> </semantics></math> on right curve. The 21 participants’ entropy values in three different scenarios (In scenario A, both the wall delineators and pavement delineators were set. In scenario B, only the pavement delineators were set. In scenario C, there were no delineators.) combined with three tunnel alignments (straight, left curve and right curve). Full article ">

28 pages, 6517 KiB

Open AccessArticle

Fast and Efficient Image Encryption Algorithm Based on Modular Addition and SPD

by Khushbu Khalid Butt, Guohui Li, Sajid Khan and Sohaib Manzoor

Entropy 2020, 22(1), 112; https://doi.org/10.3390/e22010112 - 16 Jan 2020

Cited by 10 | Viewed by 5093

Abstract

Bit-level and pixel-level methods are two classifications for image encryption, which describe the smallest processing elements manipulated in diffusion and permutation respectively. Most pixel-level permutation methods merely alter the positions of pixels, resulting in similar histograms for the original and permuted images. Bit-level [...] Read more.

Bit-level and pixel-level methods are two classifications for image encryption, which describe the smallest processing elements manipulated in diffusion and permutation respectively. Most pixel-level permutation methods merely alter the positions of pixels, resulting in similar histograms for the original and permuted images. Bit-level permutation methods, however, have the ability to change the histogram of the image, but are usually not preferred due to their time-consuming nature, which is owed to bit-level computation, unlike that of other permutation techniques. In this paper, we introduce a new image encryption algorithm which uses binary bit-plane scrambling and an SPD diffusion technique for the bit-planes of a plain image, based on a card game trick. Integer values of the hexadecimal key SHA-512 are also used, along with the adaptive block-based modular addition of pixels to encrypt the images. To prove the first-rate encryption performance of our proposed algorithm, security analyses are provided in this paper. Simulations and other results confirmed the robustness of the proposed image encryption algorithm against many well-known attacks; in particular, brute-force attacks, known/chosen plain text attacks, occlusion attacks, differential attacks, and gray value difference attacks, among others. Full article

(This article belongs to the Section Multidisciplinary Applications)

► Show Figures

Figure 1

29 pages, 970 KiB

Open AccessEditor’s ChoiceArticle

Quantifying Athermality and Quantum Induced Deviations from Classical Fluctuation Relations

by Zoë Holmes, Erick Hinds Mingo, Calvin Y.-R. Chen and Florian Mintert

Entropy 2020, 22(1), 111; https://doi.org/10.3390/e22010111 - 16 Jan 2020

Cited by 2 | Viewed by 4304

Abstract

In recent years, a quantum information theoretic framework has emerged for incorporating non-classical phenomena into fluctuation relations. Here, we elucidate this framework by exploring deviations from classical fluctuation relations resulting from the athermality of the initial thermal system and quantum coherence of the [...] Read more.

In recent years, a quantum information theoretic framework has emerged for incorporating non-classical phenomena into fluctuation relations. Here, we elucidate this framework by exploring deviations from classical fluctuation relations resulting from the athermality of the initial thermal system and quantum coherence of the system’s energy supply. In particular, we develop Crooks-like equalities for an oscillator system which is prepared either in photon added or photon subtracted thermal states and derive a Jarzynski-like equality for average work extraction. We use these equalities to discuss the extent to which adding or subtracting a photon increases the informational content of a state, thereby amplifying the suppression of free energy increasing process. We go on to derive a Crooks-like equality for an energy supply that is prepared in a pure binomial state, leading to a non-trivial contribution from energy and coherence on the resultant irreversibility. We show how the binomial state equality fits in relation to a previously derived coherent state equality and offers a richer feature-set. Full article

(This article belongs to the Special Issue Fluctuation Relations and Nonequilibrium Thermodynamics in Classical and Quantum Systems)

► Show Figures

Graphical abstract

22 pages, 4286 KiB

Open AccessArticle

Gravity Wave Activity in the Stratosphere before the 2011 Tohoku Earthquake as the Mechanism of Lithosphere-atmosphere-ionosphere Coupling

by Shih-Sian Yang and Masashi Hayakawa

Entropy 2020, 22(1), 110; https://doi.org/10.3390/e22010110 - 16 Jan 2020

Cited by 28 | Viewed by 4192

Abstract

The precursory atmospheric gravity wave (AGW) activity in the stratosphere has been investigated in our previous paper by studying an inland Kumamoto earthquake (EQ). We are interested in whether the same phenomenon occurs or not before another major EQ, especially an oceanic EQ. [...] Read more.

The precursory atmospheric gravity wave (AGW) activity in the stratosphere has been investigated in our previous paper by studying an inland Kumamoto earthquake (EQ). We are interested in whether the same phenomenon occurs or not before another major EQ, especially an oceanic EQ. In this study, we have examined the stratospheric AGW activity before the oceanic 2011 Tohoku EQ (M_w 9.0), while using the temperature profiles that were retrieved from ERA5. The potential energy (E_P) of AGW has enhanced from 3 to 7 March, 4–8 days before the EQ. The active region of the precursory AGW first appeared around the EQ epicenter, and then expanded omnidirectionally, but mainly toward the east, covering a wide area of 2500 km (in longitude) by 1500 km (in latitude). We also found the influence of the present AGW activity on some stratospheric parameters. The stratopause was heated and descended; the ozone concentration was also reduced and the zonal wind was reversed at the stratopause altitude before the EQ. These abnormalities of the stratospheric AGW and physical/chemical parameters are most significant on 5–6 March, which are found to be consistent in time and spatial distribution with the lower ionospheric perturbation, as detected by our VLF network observations. We have excluded the other probabilities by the processes of elimination and finally concluded that the abnormal phenomena observed in the present study are EQ precursors, although several potential sources can generate AGW activities and chemical variations in the stratosphere. The present paper shows that the abnormal stratospheric AGW activity has also been detected even before an oceanic EQ, and the AGW activity has obliquely propagated upward and further disturbed the lower ionosphere. This case study has provided further support to the AGW hypothesis of the lithosphere-atmosphere-ionosphere coupling process. Full article

(This article belongs to the Special Issue Entropy, Nonlinear Dynamics, and Methods of Complex Systems in Earthquake Physics including Precursory Phenomena II)

► Show Figures

Figure 1

15 pages, 3180 KiB

Open AccessArticle

Visual Analysis on Information Theory and Science of Complexity Approaches in Healthcare Research

by Xiaoyu Wang, Wang Zhao, Yongzhong Wang, Qin Zhao, Xuejie Yang, Kaixiang Su and Dongxiao Gu

Entropy 2020, 22(1), 109; https://doi.org/10.3390/e22010109 - 16 Jan 2020

Cited by 6 | Viewed by 4901

Abstract

In order to explore the knowledge base, research hotspot, development status, and future research direction of healthcare research based on information theory and complex science, a total of 3031 literature data samples from the core collection of Web of Science from 2003 to [...] Read more.

In order to explore the knowledge base, research hotspot, development status, and future research direction of healthcare research based on information theory and complex science, a total of 3031 literature data samples from the core collection of Web of Science from 2003 to 2019 were selected for bibliometric analysis. HistCite, CiteSpace, Excel, and other analytical tools were used to deeply analyze and visualize the temporal distribution, spatial distribution, knowledge evolution, literature co-citation, and research hotspots of this field. This paper reveals the current development of healthcare research field based on information theory and science of complexity, analyzes and discusses the research hotspots and future development that trends in this field, and provides important knowledge support for researchers in this field for further relevant research. Full article

(This article belongs to the Special Issue Information Theory and Complexity Science Approaches to Health Conditions and Cognitive Decline)

► Show Figures

Figure 1

32 pages, 1107 KiB

Open AccessArticle

Generalizing Information to the Evolution of Rational Belief

by Jed A. Duersch and Thomas A. Catanach

Entropy 2020, 22(1), 108; https://doi.org/10.3390/e22010108 - 16 Jan 2020

Cited by 7 | Viewed by 3441

Abstract

Information theory provides a mathematical foundation to measure uncertainty in belief. Belief is represented by a probability distribution that captures our understanding of an outcome’s plausibility. Information measures based on Shannon’s concept of entropy include realization information, Kullback–Leibler divergence, Lindley’s information in experiment, [...] Read more.

Information theory provides a mathematical foundation to measure uncertainty in belief. Belief is represented by a probability distribution that captures our understanding of an outcome’s plausibility. Information measures based on Shannon’s concept of entropy include realization information, Kullback–Leibler divergence, Lindley’s information in experiment, cross entropy, and mutual information. We derive a general theory of information from first principles that accounts for evolving belief and recovers all of these measures. Rather than simply gauging uncertainty, information is understood in this theory to measure change in belief. We may then regard entropy as the information we expect to gain upon realization of a discrete latent random variable. This theory of information is compatible with the Bayesian paradigm in which rational belief is updated as evidence becomes available. Furthermore, this theory admits novel measures of information with well-defined properties, which we explored in both analysis and experiment. This view of information illuminates the study of machine learning by allowing us to quantify information captured by a predictive model and distinguish it from residual information contained in training data. We gain related insights regarding feature selection, anomaly detection, and novel Bayesian approaches. Full article

► Show Figures

Figure 1

12 pages, 325 KiB

Open AccessArticle

On the Composability of Statistically Secure Random Oblivious Transfer

by Rafael Dowsley, Jörn Müller-Quade and Anderson C. A. Nascimento

Entropy 2020, 22(1), 107; https://doi.org/10.3390/e22010107 - 16 Jan 2020

Cited by 2 | Viewed by 2891

Abstract

We show that random oblivious transfer protocols that are statistically secure according to a definition based on a list of information-theoretical properties are also statistically universally composable. That is, they are simulatable secure with an unlimited adversary, an unlimited simulator, and an unlimited [...] Read more.

We show that random oblivious transfer protocols that are statistically secure according to a definition based on a list of information-theoretical properties are also statistically universally composable. That is, they are simulatable secure with an unlimited adversary, an unlimited simulator, and an unlimited environment machine. Our result implies that several previous oblivious transfer protocols in the literature that were proven secure under weaker, non-composable definitions of security can actually be used in arbitrary statistically secure applications without lowering the security. Full article

► Show Figures

Figure 1

15 pages, 4537 KiB

Open AccessArticle

Spatio-Temporal Evolution Analysis of Drought Based on Cloud Transformation Algorithm over Northern Anhui Province

by Xia Bai, Yimin Wang, Juliang Jin, Shaowei Ning, Yanfang Wang and Chengguo Wu

Entropy 2020, 22(1), 106; https://doi.org/10.3390/e22010106 - 16 Jan 2020

Cited by 5 | Viewed by 2799

Abstract

Drought is one of the most typical and serious natural disasters, which occurs frequently in most of mainland China, and it is crucial to explore the evolution characteristics of drought for developing effective schemes and strategies of drought disaster risk management. Based on [...] Read more.

Drought is one of the most typical and serious natural disasters, which occurs frequently in most of mainland China, and it is crucial to explore the evolution characteristics of drought for developing effective schemes and strategies of drought disaster risk management. Based on the application of Cloud theory in the drought evolution research field, the cloud transformation algorithm, and the conception zooming coupling model was proposed to re-fit the distribution pattern of SPI instead of the Pearson-III distribution. Then the spatio-temporal evolution features of drought were further summarized utilizing the cloud characteristics, average, entropy, and hyper-entropy. Lastly, the application results in Northern Anhui province revealed that the drought condition was the most serious during the period from 1957 to 1970 with the SPI12 index in 49 months being less than −0.5 and 12 months with an extreme drought level. The overall drought intensity varied with the highest certainty level but lowest stability level in winter, but this was opposite in the summer. Moreover, drought hazard would be more significantly intensified along the elevation of latitude in Northern Anhui province. The overall drought hazard in Suzhou and Huaibei were the most serious, which is followed by Bozhou, Bengbu, and Fuyang. Drought intensity in Huainan was the lightest. The exploration results of drought evolution analysis were reasonable and reliable, which would supply an effective decision-making basis for establishing drought risk management strategies. Full article

(This article belongs to the Special Issue Spatial Information Theory)

► Show Figures

Figure 1

26 pages, 2607 KiB

Open AccessArticle

Statistical Complexity Analysis of Turing Machine tapes with Fixed Algorithmic Complexity Using the Best-Order Markov Model

by Jorge M. Silva, Eduardo Pinho, Sérgio Matos and Diogo Pratas

Entropy 2020, 22(1), 105; https://doi.org/10.3390/e22010105 - 16 Jan 2020

Cited by 3 | Viewed by 4537

Abstract

Sources that generate symbolic sequences with algorithmic nature may differ in statistical complexity because they create structures that follow algorithmic schemes, rather than generating symbols from a probabilistic function assuming independence. In the case of Turing machines, this means that machines with the [...] Read more.

Sources that generate symbolic sequences with algorithmic nature may differ in statistical complexity because they create structures that follow algorithmic schemes, rather than generating symbols from a probabilistic function assuming independence. In the case of Turing machines, this means that machines with the same algorithmic complexity can create tapes with different statistical complexity. In this paper, we use a compression-based approach to measure global and local statistical complexity of specific Turing machine tapes with the same number of states and alphabet. Both measures are estimated using the best-order Markov model. For the global measure, we use the Normalized Compression (NC), while, for the local measures, we define and use normal and dynamic complexity profiles to quantify and localize lower and higher regions of statistical complexity. We assessed the validity of our methodology on synthetic and real genomic data showing that it is tolerant to increasing rates of editions and block permutations. Regarding the analysis of the tapes, we localize patterns of higher statistical complexity in two regions, for a different number of machine states. We show that these patterns are generated by a decrease of the tape’s amplitude, given the setting of small rule cycles. Additionally, we performed a comparison with a measure that uses both algorithmic and statistical approaches (BDM) for analysis of the tapes. Naturally, BDM is efficient given the algorithmic nature of the tapes. However, for a higher number of states, BDM is progressively approximated by our methodology. Finally, we provide a simple algorithm to increase the statistical complexity of a Turing machine tape while retaining the same algorithmic complexity. We supply a publicly available implementation of the algorithm in C++ language under the GPLv3 license. All results can be reproduced in full with scripts provided at the repository. Full article

(This article belongs to the Special Issue Shannon Information and Kolmogorov Complexity)

► Show Figures

Figure 1

12 pages, 474 KiB

Open AccessArticle

Complexity of Cardiotocographic Signals as A Predictor of Labor

by João Monteiro-Santos, Teresa Henriques, Inês Nunes, Célia Amorim-Costa, João Bernardes and Cristina Costa-Santos

Entropy 2020, 22(1), 104; https://doi.org/10.3390/e22010104 - 16 Jan 2020

Cited by 5 | Viewed by 3370

Abstract

Prediction of labor is of extreme importance in obstetric care to allow for preventive measures, assuring that both baby and mother have the best possible care. In this work, the authors studied how important nonlinear parameters (entropy and compression) can be as labor [...] Read more.

Prediction of labor is of extreme importance in obstetric care to allow for preventive measures, assuring that both baby and mother have the best possible care. In this work, the authors studied how important nonlinear parameters (entropy and compression) can be as labor predictors. Linear features retrieved from the SisPorto system for cardiotocogram analysis and nonlinear measures were used to predict labor in a dataset of 1072 antepartum tracings, at between 30 and 35 weeks of gestation. Two groups were defined: Group A—fetuses whose traces date was less than one or two weeks before labor, and Group B—fetuses whose traces date was at least one or two weeks before labor. Results suggest that, compared with linear features such as decelerations and variability indices, compression improves labor prediction both within one (C-Statistics of 0.728) and two weeks (C-Statistics of 0.704). Moreover, the correlation between compression and long-term variability was significantly different in groups A and B, denoting that compression and heart rate variability look at different information associated with whether the fetus is closer to or further from labor onset. Nonlinear measures, compression in particular, may be useful in improving labor prediction as a complement to other fetal heart rate features. Full article

(This article belongs to the Special Issue Assessing Complexity in Physiological Systems through Biomedical Signals Analysis)

► Show Figures

Figure 1

14 pages, 3812 KiB

Open AccessEditor’s ChoiceArticle

Determining the Bulk Parameters of Plasma Electrons from Pitch-Angle Distribution Measurements

by Georgios Nicolaou, Robert Wicks, George Livadiotis, Daniel Verscharen, Christopher Owen and Dhiren Kataria

Entropy 2020, 22(1), 103; https://doi.org/10.3390/e22010103 - 16 Jan 2020

Cited by 15 | Viewed by 4878

Abstract

Electrostatic analysers measure the flux of plasma particles in velocity space and determine their velocity distribution function. There are occasions when science objectives require high time-resolution measurements, and the instrument operates in short measurement cycles, sampling only a portion of the velocity distribution [...] Read more.

Electrostatic analysers measure the flux of plasma particles in velocity space and determine their velocity distribution function. There are occasions when science objectives require high time-resolution measurements, and the instrument operates in short measurement cycles, sampling only a portion of the velocity distribution function. One such high-resolution measurement strategy consists of sampling the two-dimensional pitch-angle distributions of the plasma particles, which describes the velocities of the particles with respect to the local magnetic field direction. Here, we investigate the accuracy of plasma bulk parameters from such high-resolution measurements. We simulate electron observations from the Solar Wind Analyser’s (SWA) Electron Analyser System (EAS) on board Solar Orbiter. We show that fitting analysis of the synthetic datasets determines the plasma temperature and kappa index of the distribution within 10% of their actual values, even at large heliocentric distances where the expected solar wind flux is very low. Interestingly, we show that although measurement points with zero counts are not statistically significant, they provide information about the particle distribution function which becomes important when the particle flux is low. We also examine the convergence of the fitting algorithm for expected plasma conditions and discuss the sources of statistical and systematic uncertainties. Full article

(This article belongs to the Special Issue Theoretical Aspects of Kappa Distributions)

► Show Figures

Figure 1

19 pages, 2322 KiB

Open AccessFeature PaperEditor’s ChoiceArticle

Learning in Feedforward Neural Networks Accelerated by Transfer Entropy

by Adrian Moldovan, Angel Caţaron and Răzvan Andonie

Entropy 2020, 22(1), 102; https://doi.org/10.3390/e22010102 - 16 Jan 2020

Cited by 18 | Viewed by 5464

Abstract

Current neural networks architectures are many times harder to train because of the increasing size and complexity of the used datasets. Our objective is to design more efficient training algorithms utilizing causal relationships inferred from neural networks. The transfer entropy (TE) was initially [...] Read more.

Current neural networks architectures are many times harder to train because of the increasing size and complexity of the used datasets. Our objective is to design more efficient training algorithms utilizing causal relationships inferred from neural networks. The transfer entropy (TE) was initially introduced as an information transfer measure used to quantify the statistical coherence between events (time series). Later, it was related to causality, even if they are not the same. There are only few papers reporting applications of causality or TE in neural networks. Our contribution is an information-theoretical method for analyzing information transfer between the nodes of feedforward neural networks. The information transfer is measured by the TE of feedback neural connections. Intuitively, TE measures the relevance of a connection in the network and the feedback amplifies this connection. We introduce a backpropagation type training algorithm that uses TE feedback connections to improve its performance. Full article

(This article belongs to the Section Information Theory, Probability and Statistics)

► Show Figures

Figure 1

7 pages, 276 KiB

Open AccessArticle

A Geometric Interpretation of Stochastic Gradient Descent Using Diffusion Metrics

by Rita Fioresi, Pratik Chaudhari and Stefano Soatto

Entropy 2020, 22(1), 101; https://doi.org/10.3390/e22010101 - 15 Jan 2020

Cited by 3 | Viewed by 5483

Abstract

This paper is a step towards developing a geometric understanding of a popular algorithm for training deep neural networks named stochastic gradient descent (SGD). We built upon a recent result which observed that the noise in SGD while training typical networks is highly [...] Read more.

This paper is a step towards developing a geometric understanding of a popular algorithm for training deep neural networks named stochastic gradient descent (SGD). We built upon a recent result which observed that the noise in SGD while training typical networks is highly non-isotropic. That motivated a deterministic model in which the trajectories of our dynamical systems are described via geodesics of a family of metrics arising from a certain diffusion matrix; namely, the covariance of the stochastic gradients in SGD. Our model is analogous to models in general relativity: the role of the electromagnetic field in the latter is played by the gradient of the loss function of a deep network in the former. Full article

(This article belongs to the Special Issue The Information Bottleneck in Deep Learning)

13 pages, 5513 KiB

Open AccessEditor’s ChoiceArticle

On Heat Transfer Performance of Cooling Systems Using Nanofluid for Electric Motor Applications

by Ali Deriszadeh and Filippo de Monte

Entropy 2020, 22(1), 99; https://doi.org/10.3390/e22010099 - 14 Jan 2020

Cited by 29 | Viewed by 7804

Abstract

This paper studies the fluid flow and heat transfer characteristics of nanofluids as advance coolants for the cooling system of electric motors. Investigations are carried out using numerical analysis for a cooling system with spiral channels. To solve the governing equations, computational fluid [...] Read more.

This paper studies the fluid flow and heat transfer characteristics of nanofluids as advance coolants for the cooling system of electric motors. Investigations are carried out using numerical analysis for a cooling system with spiral channels. To solve the governing equations, computational fluid dynamics and 3D fluid motion analysis are used. The base fluid is water with a laminar flow. The fluid Reynolds number and turn-number of spiral channels are evaluation parameters. The effect of nanoparticles volume fraction in the base fluid on the heat transfer performance of the cooling system is studied. Increasing the volume fraction of nanoparticles leads to improving the heat transfer performance of the cooling system. On the other hand, a high-volume fraction of the nanofluid increases the pressure drop of the coolant fluid and increases the required pumping power. This paper aims at finding a trade-off between effective parameters by studying both fluid flow and heat transfer characteristics of the nanofluid. Full article

(This article belongs to the Special Issue Selected Papers from the XII International Conference on Computational Heat, Mass and Momentum Transfer (ICCHMT2019))

► Show Figures

Figure 1

27 pages, 1642 KiB

Open AccessEditor’s ChoiceArticle

The Convex Information Bottleneck Lagrangian

by Borja Rodríguez Gálvez, Ragnar Thobaben and Mikael Skoglund

Entropy 2020, 22(1), 98; https://doi.org/10.3390/e22010098 - 14 Jan 2020

Cited by 20 | Viewed by 5201

Abstract

The information bottleneck (IB) problem tackles the issue of obtaining relevant compressed representations T of some random variable X for the task of predicting Y. It is defined as a constrained optimization problem that maximizes the information the representation has about the [...] Read more.

The information bottleneck (IB) problem tackles the issue of obtaining relevant compressed representations T of some random variable X for the task of predicting Y. It is defined as a constrained optimization problem that maximizes the information the representation has about the task,

I (T; Y)

, while ensuring that a certain level of compression r is achieved (i.e.,

I (X; T) \leq r

). For practical reasons, the problem is usually solved by maximizing the IB Lagrangian (i.e.,

L_{IB} (T; β) = I (T; Y) - β I (X; T)

) for many values of

β \in [0, 1]

. Then, the curve of maximal

I (T; Y)

for a given

I (X; T)

is drawn and a representation with the desired predictability and compression is selected. It is known when Y is a deterministic function of X, the IB curve cannot be explored and another Lagrangian has been proposed to tackle this problem: the squared IB Lagrangian:

L_{sq - IB} (T; β_{sq}) = I (T; Y) - β_{sq} I {(X; T)}^{2}

. In this paper, we (i) present a general family of Lagrangians which allow for the exploration of the IB curve in all scenarios; (ii) provide the exact one-to-one mapping between the Lagrange multiplier and the desired compression rate r for known IB curve shapes; and (iii) show we can approximately obtain a specific compression level with the convex IB Lagrangian for both known and unknown IB curve shapes. This eliminates the burden of solving the optimization problem for many values of the Lagrange multiplier. That is, we prove that we can solve the original constrained problem with a single optimization. Full article

(This article belongs to the Special Issue Information Bottleneck: Theory and Applications in Deep Learning)

► Show Figures

Graphical abstract

20 pages, 1424 KiB

Open AccessReview

A Review of the Application of Information Theory to Clinical Diagnostic Testing

by William A. Benish

Entropy 2020, 22(1), 97; https://doi.org/10.3390/e22010097 - 14 Jan 2020

Cited by 14 | Viewed by 5418

Abstract

The fundamental information theory functions of entropy, relative entropy, and mutual information are directly applicable to clinical diagnostic testing. This is a consequence of the fact that an individual’s disease state and diagnostic test result are random variables. In this paper, we review [...] Read more.

The fundamental information theory functions of entropy, relative entropy, and mutual information are directly applicable to clinical diagnostic testing. This is a consequence of the fact that an individual’s disease state and diagnostic test result are random variables. In this paper, we review the application of information theory to the quantification of diagnostic uncertainty, diagnostic information, and diagnostic test performance. An advantage of information theory functions over more established test performance measures is that they can be used when multiple disease states are under consideration as well as when the diagnostic test can yield multiple or continuous results. Since more than one diagnostic test is often required to help determine a patient’s disease state, we also discuss the application of the theory to situations in which more than one diagnostic test is used. The total diagnostic information provided by two or more tests can be partitioned into meaningful components. Full article

(This article belongs to the Special Issue Applications of Information Theory to Epidemiology)

► Show Figures

Figure 1

Figure 1
Surprisal (in bits) as a function of probability. Full article ">Figure 2
The area under the receiver operating characteristic curve (AUC), a transformation of the AUC (AUC*), and the mutual information between the disease state and the test result (<math display="inline"><semantics> <mrow> <mi>I</mi> <mrow> <mo>(</mo> <mrow> <mi>D</mi> <mo>;</mo> <mi>R</mi> </mrow> <mo>)</mo> </mrow> </mrow> </semantics></math> in units of bits), as a function of the distance between the means of the distributions of test results for both healthy (d<math display="inline"><semantics> <mo>−</mo> </semantics></math>) and diseased (d<math display="inline"><semantics> <mo>+</mo> </semantics></math>) individuals (see text). <math display="inline"><semantics> <mrow> <mi>I</mi> <mrow> <mo>(</mo> <mrow> <mi>D</mi> <mo>;</mo> <mi>R</mi> </mrow> <mo>)</mo> </mrow> </mrow> </semantics></math> is plotted for three pretest probabilities of disease: 0.1, 0.2, and 0.5. Full article ">Figure 3
Mutual information as a function of pretest probability of significant coronary heart disease (CHD) for the exercise electrocardiogram. The plot compares the performance of a theoretical ideal test with the actual performance when the results are either (1) dichotomized using the criterion of ST segment depression of <math display="inline"><semantics> <mo>≥</mo> </semantics></math> 1 mm or (2) not dichotomized. This plot has been reconstructed with permission from the paper by Diamond et al. [<a href="#B6-entropy-22-00097" class="html-bibr">6</a>]. Full article ">Figure 4
Diagnostic information (relative entropy) provided by the findings of an ST segment depression (STdep) < 0.5 mm and an ST depression (STdep) ≥ 2.5 mm as a function of pretest probability of significant coronary heart disease (CHD). Also shown are relative entropy plots for a theoretical ideal test when significant CHD is present (d+) and when significant CHD is absent (d<math display="inline"><semantics> <mo>−</mo> </semantics></math>). For the theoretical ideal test when significant CHD is present, relative entropy increases indefinitely as pretest probability of significant CHD approaches zero; and for the theoretical ideal test when significant CHD is absent, relative entropy increases indefinitely as pretest probability of significant CHD approaches one. Full article ">Figure 5
Contour plots showing the probability densities of the results of tests A and B for the healthy and diseased populations. The test results are expressed as standard deviations from the means of the distribution for healthy patients <math display="inline"><semantics> <mrow> <mrow> <mo>(</mo> <mrow> <msub> <mi>μ</mi> <mi>A</mi> </msub> <mo>=</mo> <msub> <mi>μ</mi> <mi>B</mi> </msub> <mo>=</mo> <mn>0</mn> </mrow> <mo>)</mo> </mrow> </mrow> </semantics></math>. Full article ">Figure 6
Contour plot showing the information (relative entropy in units of bits) provided by specific combinations of results of tests A and B. The test results are expressed as standard deviations from the means of the distribution for healthy patients <math display="inline"><semantics> <mrow> <mrow> <mo>(</mo> <mrow> <msub> <mi>μ</mi> <mi>A</mi> </msub> <mo>=</mo> <msub> <mi>μ</mi> <mi>B</mi> </msub> <mo>=</mo> <mn>0</mn> </mrow> <mo>)</mo> </mrow> </mrow> </semantics></math>. Full article ">

Journal Menu

Journal Browser

Entropy, Volume 22, Issue 1 (January 2020) – 127 articles

Further Information

Guidelines

MDPI Initiatives

Follow MDPI