Entropy

620 KiB

Open AccessArticle

The Quantum-Classical Transition as an Information Flow

by Andres M. Kowalski, Maria T. Martin, Luciano Zunino, Angelo Plastino and Montserrat Casas

Entropy 2010, 12(1), 148-160; https://doi.org/10.3390/e12010148 - 26 Jan 2010

Cited by 5 | Viewed by 9591

We investigate the classical limit of the semiclassical evolution with reference to a well-known model that represents the interaction between matter and a given field. This is done by recourse to a special statistical quantifier called the “symbolic transfer entropy”. We encounter that [...] Read more.

We investigate the classical limit of the semiclassical evolution with reference to a well-known model that represents the interaction between matter and a given field. This is done by recourse to a special statistical quantifier called the “symbolic transfer entropy”. We encounter that the quantum-classical transition gets thereby described as the sign-reversal of the dominating direction of the information flow between classical and quantal variables. Full article

(This article belongs to the Special Issue Information and Entropy)

► Show Figures

Figure 1

488 KiB

Open AccessArticle

On the Entropy Based Associative Memory Model with Higher-Order Correlations

by Masahiro Nakagawa

Entropy 2010, 12(1), 136-147; https://doi.org/10.3390/e12010136 - 22 Jan 2010

Cited by 1 | Viewed by 7464

Abstract

In this paper, an entropy based associative memory model will be proposed and applied to memory retrievals with an orthogonal learning model so as to compare with the conventional model based on the quadratic Lyapunov functional to be minimized during the retrieval process. [...] Read more.

In this paper, an entropy based associative memory model will be proposed and applied to memory retrievals with an orthogonal learning model so as to compare with the conventional model based on the quadratic Lyapunov functional to be minimized during the retrieval process. In the present approach, the updating dynamics will be constructed on the basis of the entropy minimization strategy which may be reduced asymptotically to the above-mentioned conventional dynamics as a special case ignoring the higher-order correlations. According to the introduction of the entropy functional, one may involve higer-order correlation effects between neurons in a self-contained manner without any heuristic coupling coefficients as in the conventional manner. In fact we shall show such higher order coupling tensors are to be uniquely determined in the framework of the entropy based approach. From numerical results, it will be found that the presently proposed novel approach realizes much larger memory capacity than that of the quadratic Lyapunov functional approach, e.g., associatron. Full article

(This article belongs to the Special Issue Entropy in Model Reduction)

► Show Figures

Figure 1

Figure 1
The time dependence of overlaps of the present entropy based model defined by Equation (17). Full article ">Figure 2
The time dependence of overlaps of the associatron defined by Equation (18). Full article ">Figure 3
The dependence of the success rate on the loading rate α = L/N of autoassociation model based on Equation (17) (entropy based approach). Full article ">Figure 3 Cont.
The dependence of the success rate on the loading rate α = L/N of autoassociation model based on Equation (17) (entropy based approach). Full article ">Figure 4
The dependence of the success rate on the loading rate α = L/N of autoassociation model based on Equation (18) (associatron). Full article ">Figure 5
The dependence of the storage capacity on the Hamming distance. Here symbols o and x are for the entropy based approach and the associatron, respectively. Full article ">

98 KiB

Open AccessArticle

Arguments for the Integration of the Non-Zero-Sum Logic of Complex Animal Communication with Information Theory

by Vincenzo Penteriani

Entropy 2010, 12(1), 127-135; https://doi.org/10.3390/e12010127 - 21 Jan 2010

Cited by 2 | Viewed by 8028

Abstract

The outstanding levels of knowledge attained today in the research on animal communication, and the new available technologies to study visual, vocal and chemical signalling, allow an ever increasing use of information theory as a sophisticated tool to improve our knowledge of the [...] Read more.

The outstanding levels of knowledge attained today in the research on animal communication, and the new available technologies to study visual, vocal and chemical signalling, allow an ever increasing use of information theory as a sophisticated tool to improve our knowledge of the complexity of animal communication. Some considerations on the way information theory and intraspecific communication can be linked are presented here. Specifically, information theory may help us to explore interindividual variations in different environmental constraints and social scenarios, as well as the communicative features of social vs. solitary species. Full article

(This article belongs to the Special Issue Information Theory Applied to Animal Communication)

380 KiB

Open AccessArticle

From Maximum Entropy to Maximum Entropy Production: A New Approach

by Nathaniel Virgo

Entropy 2010, 12(1), 107-126; https://doi.org/10.3390/e12010107 - 18 Jan 2010

Cited by 17 | Viewed by 11329

Abstract

Evidence from climate science suggests that a principle of maximum thermodynamic entropy production can be used to make predictions about some physical systems. I discuss the general form of this principle and an inherent problem with it, currently unsolved by theoretical approaches: how [...] Read more.

Evidence from climate science suggests that a principle of maximum thermodynamic entropy production can be used to make predictions about some physical systems. I discuss the general form of this principle and an inherent problem with it, currently unsolved by theoretical approaches: how to determine which system it should be applied to. I suggest a new way to derive the principle from statistical mechanics, and present a tentative solution to the system boundary problem. I discuss the need for experimental validation of the principle, and its impact on the way we see the relationship between thermodynamics and kinetics. Full article

(This article belongs to the Special Issue What Is Maximum Entropy Production and How Should We Apply It?)

► Show Figures

Graphical abstract

284 KiB

Open AccessReview

Maximum Entropy Approaches to Living Neural Networks

by Fang-Chin Yeh, Aonan Tang, Jon P. Hobbs, Pawel Hottowy, Wladyslaw Dabrowski, Alexander Sher, Alan Litke and John M. Beggs

Entropy 2010, 12(1), 89-106; https://doi.org/10.3390/e12010089 - 13 Jan 2010

Cited by 40 | Viewed by 12180

Abstract

Understanding how ensembles of neurons collectively interact will be a key step in developing a mechanistic theory of cognitive processes. Recent progress in multineuron recording and analysis techniques has generated tremendous excitement over the physiology of living neural networks. One of the key [...] Read more.

Understanding how ensembles of neurons collectively interact will be a key step in developing a mechanistic theory of cognitive processes. Recent progress in multineuron recording and analysis techniques has generated tremendous excitement over the physiology of living neural networks. One of the key developments driving this interest is a new class of models based on the principle of maximum entropy. Maximum entropy models have been reported to account for spatial correlation structure in ensembles of neurons recorded from several different types of data. Importantly, these models require only information about the firing rates of individual neurons and their pairwise correlations. If this approach is generally applicable, it would drastically simplify the problem of understanding how neural networks behave. Given the interest in this method, several groups now have worked to extend maximum entropy models to account for temporal correlations. Here, we review how maximum entropy models have been applied to neuronal ensemble data to account for spatial and temporal correlations. We also discuss criticisms of the maximum entropy approach that argue that it is not generally applicable to larger ensembles of neurons. We conclude that future maximum entropy models will need to address three issues: temporal correlations, higher-order correlations, and larger ensemble sizes. Finally, we provide a brief list of topics for future research. Full article

► Show Figures

Figure 1

Figure 1
The problem to be solved. Full article ">Figure 2
Temporal correlations are important. (a) Activity from many neurons plotted over time. Boxes highlight an ensemble of four neurons over six time bins. (b) Within the boxes, there was activity for four consecutive time bins, bracketed by no activity at the beginning and the end. This is a sequence of length L = 4 (see text). (c) Sequence length distributions from actual data were significantly longer than those produced by random concatenations of states from the model. This suggests that temporal correlations play an important part in determining activity in neuronal ensembles. Full article ">Figure 3
Accounting for temporal correlations. Full article ">Figure 4
Distribution of states for a model with temporal correlations (a) An ensemble of four neurons is selected from the raster plot. (b) Here, activity over a span of three time bins (t, t + 1, and t + 2) is considered one state. (c) The distribution of states is plotted for the model and for the data. Full article ">Figure 5
Quantifying temporal correlations. Full article ">

158 KiB

Open AccessArticle

A Dynamic Model of Information and Entropy

by Michael C. Parker and Stuart D. Walker

Entropy 2010, 12(1), 80-88; https://doi.org/10.3390/e12010080 - 7 Jan 2010

Cited by 9 | Viewed by 7903

Abstract

We discuss the possibility of a relativistic relationship between information and entropy, closely analogous to the classical Maxwell electro-magnetic wave equations. Inherent to the analysis is the description of information as residing in points of non-analyticity; yet ultimately also exhibiting a distributed characteristic: [...] Read more.

We discuss the possibility of a relativistic relationship between information and entropy, closely analogous to the classical Maxwell electro-magnetic wave equations. Inherent to the analysis is the description of information as residing in points of non-analyticity; yet ultimately also exhibiting a distributed characteristic: additionally analogous, therefore, to the wave-particle duality of light. At cosmological scales our vector differential equations predict conservation of information in black holes, whereas regular- and Z-DNA molecules correspond to helical solutions at microscopic levels. We further propose that regular- and Z-DNA are equivalent to the alternative words chosen from an alphabet to maintain the equilibrium of an information transmission system. Full article

(This article belongs to the Special Issue Information and Entropy)

► Show Figures

Figure 1

637 KiB

Open AccessArticle

Redundancy in Systems Which Entertain a Model of Themselves: Interaction Information and the Self-Organization of Anticipation

by Loet Leydesdorff

Entropy 2010, 12(1), 63-79; https://doi.org/10.3390/e12010063 - 6 Jan 2010

Cited by 25 | Viewed by 12402

Abstract

Mutual information among three or more dimensions (μ* = –Q) has been considered as interaction information. However, Krippendorff [1,2] has shown that this measure cannot be interpreted as a unique property of the interactions and has proposed an alternative measure of interaction information [...] Read more.

Mutual information among three or more dimensions (μ* = –Q) has been considered as interaction information. However, Krippendorff [1,2] has shown that this measure cannot be interpreted as a unique property of the interactions and has proposed an alternative measure of interaction information based on iterative approximation of maximum entropies. Q can then be considered as a measure of the difference between interaction information and redundancy generated in a model entertained by an observer. I argue that this provides us with a measure of the imprint of a second-order observing system—a model entertained by the system itself—on the underlying information processing. The second-order system communicates meaning hyper-incursively; an observation instantiates this meaning-processing within the information processing. The net results may add to or reduce the prevailing uncertainty. The model is tested empirically for the case where textual organization can be expected to contain intellectual organization in terms of distributions of title words, author names, and cited references. Full article

(This article belongs to the Special Issue Information and Entropy)

► Show Figures

Figure 1

217 KiB

Open AccessArticle

Imprecise Shannon’s Entropy and Multi Attribute Decision Making

by Farhad Hosseinzadeh Lotfi and Reza Fallahnejad

Entropy 2010, 12(1), 53-62; https://doi.org/10.3390/e12010053 - 5 Jan 2010

Cited by 300 | Viewed by 16100

Abstract

Finding the appropriate weight for each criterion is one of the main points in Multi Attribute Decision Making (MADM) problems. Shannon’s entropy method is one of the various methods for finding weights discussed in the literature. However, in many real life problems, the [...] Read more.

Finding the appropriate weight for each criterion is one of the main points in Multi Attribute Decision Making (MADM) problems. Shannon’s entropy method is one of the various methods for finding weights discussed in the literature. However, in many real life problems, the data for the decision making processes cannot be measured precisely and there may be some other types of data, for instance, interval data and fuzzy data. The goal of this paper is the extension of the Shannon entropy method for the imprecise data, especially interval and fuzzy data cases. Full article

435 KiB

Open AccessReview

Data Compression Concepts and Algorithms and Their Applications to Bioinformatics

by Özkan U. Nalbantoglu, David J. Russell and Khalid Sayood

Entropy 2010, 12(1), 34-52; https://doi.org/10.3390/e12010034 - 29 Dec 2009

Cited by 37 | Viewed by 10885

Abstract

Data compression at its base is concerned with how information is organized in data. Understanding this organization can lead to efficient ways of representing the information and hence data compression. In this paper we review the ways in which ideas and approaches fundamental [...] Read more.

Data compression at its base is concerned with how information is organized in data. Understanding this organization can lead to efficient ways of representing the information and hence data compression. In this paper we review the ways in which ideas and approaches fundamental to the theory and practice of data compression have been used in the area of bioinformatics. We look at how basic theoretical ideas from data compression, such as the notions of entropy, mutual information, and complexity have been used for analyzing biological sequences in order to discover hidden patterns, infer phylogenetic relationships between organisms and study viral populations. Finally, we look at how inferred grammars for biological sequences have been used to uncover structure in biological sequences. Full article

► Show Figures

Figure 1

Figure 1
Plot of the redundancy rate versus <math display="inline"> <mfrac> <msub> <mi>D</mi> <mn>2</mn> </msub> <mrow> <msub> <mi>D</mi> <mn>1</mn> </msub> <mo>+</mo> <msub> <mi>D</mi> <mn>2</mn> </msub> </mrow> </mfrac> </math> using the genes available at the time shows a clear segregation of Phage, Bacteria, and Vertebrate sequences. Full article ">Figure 2
Inclusion of additional sequences breaks down the segregation observed by Gatlin. Full article ">Figure 3
The logo of a number of sequences at the beginning of a gene. The start codon ATG is immediately apparent. The logo was constructed using the software at <a href="http://weblogo.threeplusone.com/" target="_blank">http://weblogo.threeplusone.com/</a>. Full article ">Figure 4
AMI charts for HIV-1 populations isolated from patients who remained asymptomatic. The large number of white pixels indicate generally a high degree of covariation while “checkerboard" regions indicate specific segments of the envelope protein with correlated mutations [<a href="#B20-entropy-12-00034" class="html-bibr">20</a>]. Full article ">Figure 5
AMI charts for HIV-1 populations isolated from patients who succumbed to AIDS. The preponderance of black pixels indicates a relatively homogeneous population [<a href="#B20-entropy-12-00034" class="html-bibr">20</a>]. Full article ">Figure 6
A block diagram depicting the basic steps involved with a grammar-based compression scheme. Full article ">

332 KiB

Open AccessArticle

Estimation of Seismic Wavelets Based on the Multivariate Scale Mixture of Gaussians Model

by Jing-Huai Gao and Bing Zhang

Entropy 2010, 12(1), 14-33; https://doi.org/10.3390/e12010014 - 28 Dec 2009

Cited by 15 | Viewed by 8626

Abstract

This paper proposes a new method for estimating seismic wavelets. Suppose a seismic wavelet can be modeled by a formula with three free parameters (scale, frequency and phase). We can transform the estimation of the wavelet into determining these three parameters. The phase [...] Read more.

This paper proposes a new method for estimating seismic wavelets. Suppose a seismic wavelet can be modeled by a formula with three free parameters (scale, frequency and phase). We can transform the estimation of the wavelet into determining these three parameters. The phase of the wavelet is estimated by constant-phase rotation to the seismic signal, while the other two parameters are obtained by the Higher-order Statistics (HOS) (fourth-order cumulant) matching method. In order to derive the estimator of the Higher-order Statistics (HOS), the multivariate scale mixture of Gaussians (MSMG) model is applied to formulating the multivariate joint probability density function (PDF) of the seismic signal. By this way, we can represent HOS as a polynomial function of second-order statistics to improve the anti-noise performance and accuracy. In addition, the proposed method can work well for short time series. Full article

(This article belongs to the Special Issue Maximum Entropy)

► Show Figures

Figure 1

Figure 1
The error behavior of ODCR with different orders. Full article ">Figure 2
The change of the performance in variation with wavelet frequency. Full article ">Figure 3
The error performance of <math display="inline"> <semantics> <mover accent="true"> <mi>r</mi> <mo stretchy="false">^</mo> </mover> </semantics> </math>y(4)(yn(2), yn+m(2)) and <math display="inline"> <semantics> <mover accent="true"> <mi>r</mi> <mo stretchy="false">^</mo> </mover> </semantics> </math>y0(4)(yn(2), yn+m(2)). Full article ">Figure 4
(a) Super-Gaussian reflectivity sequence. (b) Reflectivity histogram. (c) Noisy seismic trace. (d) The kurtosis value of the rotated trace versus the rotation angle (the angle is represented as the multiple of π). (e) Cost surface. (f) Wavelet. Full article ">Figure 5
Real-data experiment. Full article ">

5023 KiB

Open AccessArticle

Lorenz Curves, Size Classification, and Dimensions of Bubble Size Distributions

by Sonja Sauerbrei

Entropy 2010, 12(1), 1-13; https://doi.org/10.3390/e12010001 - 25 Dec 2009

Cited by 5 | Viewed by 10125

Abstract

Lorenz curves of bubble size distributions and their Gini coefficients characterize demixing processes. Through a systematic size classification, bubble size histograms are generated and investigated concerning their statistical entropy. It turns out that the temporal development of the entropy is preserved although characteristics [...] Read more.

Lorenz curves of bubble size distributions and their Gini coefficients characterize demixing processes. Through a systematic size classification, bubble size histograms are generated and investigated concerning their statistical entropy. It turns out that the temporal development of the entropy is preserved although characteristics of the histograms like number of size classes and modality are remarkably reduced. Examinations by Rényi dimensions show that the bubble size distributions are multifractal and provide information about the underlying structures like self-similarity. Full article

► Show Figures

Figure 1

Journal Menu

Journal Browser

Entropy, Volume 12, Issue 1 (January 2010) – 11 articles , Pages 1-160

Further Information

Guidelines

MDPI Initiatives

Follow MDPI