Entropy

12 pages, 514 KiB

Open AccessArticle

New Variable-Weight Optical Orthogonal Codes with Weights 3 to 5

by Si-Yeon Pak, Hyo-Won Kim, DaeHan Ahn and Jin-Ho Chung

Entropy 2024, 26(11), 982; https://doi.org/10.3390/e26110982 (registering DOI) - 15 Nov 2024

In optical networks, designing optical orthogonal codes (OOCs) with appropriate parameters is essential for enhancing the overall system performance. They are divided into two categories, constant-weight OOCs (CW-OOCs) and variable-weight OOCs (VW-OOCs), based on the number of distinct Hamming weights present in their [...] Read more.

In optical networks, designing optical orthogonal codes (OOCs) with appropriate parameters is essential for enhancing the overall system performance. They are divided into two categories, constant-weight OOCs (CW-OOCs) and variable-weight OOCs (VW-OOCs), based on the number of distinct Hamming weights present in their codewords. This paper introduces a method for constructing VW-OOCs of length by using the structure of an integer ring and the Chinese Remainder Theorem. In particular, we present some specific VW-OOCs with weights of 3, 4, or 5. The results demonstrate that certain optimal VW-OOCs can be obtained with parameters that are not covered in the existing literature. Full article

(This article belongs to the Special Issue New Advances in Error-Correcting Codes)

50 pages, 729 KiB

Open AccessArticle

Non-Equilibrium Quantum Brain Dynamics: Water Coupled with Phonons and Photons

by Akihiro Nishiyama, Shigenori Tanaka and Jack Adam Tuszynski

Entropy 2024, 26(11), 981; https://doi.org/10.3390/e26110981 (registering DOI) - 15 Nov 2024

Abstract

We investigate Quantum Electrodynamics (QED) of water coupled with sound and light, namely Quantum Brain Dynamics (QBD) of water, phonons and photons. We provide phonon degrees of freedom as additional quanta in the framework of QBD in this paper. We begin with the [...] Read more.

We investigate Quantum Electrodynamics (QED) of water coupled with sound and light, namely Quantum Brain Dynamics (QBD) of water, phonons and photons. We provide phonon degrees of freedom as additional quanta in the framework of QBD in this paper. We begin with the Lagrangian density QED with non-relativistic charged bosons, photons and phonons, and derive time-evolution equations of coherent fields and Kadanoff–Baym (KB) equations for incoherent particles. We next show an acoustic super-radiance solution in our model. We also introduce a kinetic entropy current in KB equations in 1st order approximation in the gradient expansion and show the H-theorem for self-energy in Hartree–Fock approximation. We finally derive conserved number density of charged bosons and conserved energy density in spatially homogeneous system. Full article

(This article belongs to the Section Quantum Information)

15 pages, 27235 KiB

Open AccessArticle

Dynamics of Aggregation in Systems of Self-Propelled Rods

by Richard J. G. Löffler and Jerzy Gorecki

Entropy 2024, 26(11), 980; https://doi.org/10.3390/e26110980 - 15 Nov 2024

Abstract

We highlight camphene–camphor–polypropylene plastic as a useful material for self-propelled objects that show aggregation while floating on a water surface. We consider self-propelled rods as an example of aggregation of objects characterized by non-trivial individual shapes with low-symmetry interactions between them. The motion [...] Read more.

We highlight camphene–camphor–polypropylene plastic as a useful material for self-propelled objects that show aggregation while floating on a water surface. We consider self-propelled rods as an example of aggregation of objects characterized by non-trivial individual shapes with low-symmetry interactions between them. The motion of rods made of the camphene–camphor–polypropylene plastic is supported by dissipation of the surface-active molecules. The physical processes leading to aggregation and the mathematical model of the process are discussed. We analyze experimental data of aggregate formation dynamics and relate them to the system’s properties. We speculate that the aggregate structure can be represented as a string of symbols, which opens the potential applicability of the phenomenon for information processing if objects floating on a water surface are regarded as reservoir computers. Full article

(This article belongs to the Special Issue Matter-Aggregating Systems at a Classical vs. Quantum Interface)

► Show Figures

Figure 1

23 pages, 324 KiB

Open AccessArticle

Bowen’s Formula for a Dynamical Solenoid

by Andrzej Biś, Wojciech Kozłowski and Agnieszka Marczuk

Entropy 2024, 26(11), 979; https://doi.org/10.3390/e26110979 - 15 Nov 2024

Viewed by 122

Abstract

More than 50 years ago, Rufus Bowen noticed a natural relation between the ergodic theory and the dimension theory of dynamical systems. He proved a formula, known today as the Bowen’s formula, that relates the Hausdorff dimension of a conformal repeller to the [...] Read more.

More than 50 years ago, Rufus Bowen noticed a natural relation between the ergodic theory and the dimension theory of dynamical systems. He proved a formula, known today as the Bowen’s formula, that relates the Hausdorff dimension of a conformal repeller to the zero of a pressure function defined by a single conformal map. In this paper, we extend the result of Bowen to a sequence of conformal maps. We present a dynamical solenoid, i.e., a generalized dynamical system obtained by backward compositions of a sequence of continuous surjections

{(f_{n} : X \to X)}_{n \in N}

defined on a compact metric space

(X, d) .

Under mild assumptions, we provide a self-contained proof that Bowen’s formula holds for dynamical conformal solenoids. As a corollary, we obtain that the Bowen’s formula holds for a conformal surjection

f : X \to X

of a compact Full article

(This article belongs to the Section Statistical Physics)

13 pages, 284 KiB

Open AccessArticle

Quantum Control Design by Lyapunov Trajectory Tracking and Optimal Control

by Hongli Yang, Guohui Yu and Ivan Ganchev Ivanov

Entropy 2024, 26(11), 978; https://doi.org/10.3390/e26110978 - 15 Nov 2024

Viewed by 188

Abstract

In this paper, we investigate a Lyapunov trajectory tracking design method that incorporates a Schrödinger equation with a dipole subterm and polarizability. Our findings suggest that the proposed control law can overcome the limitations of certain existing control laws that do not converge. [...] Read more.

In this paper, we investigate a Lyapunov trajectory tracking design method that incorporates a Schrödinger equation with a dipole subterm and polarizability. Our findings suggest that the proposed control law can overcome the limitations of certain existing control laws that do not converge. By integrating a quadratic performance index, we introduce an optimal control law, which we subsequently analyze for stability and optimality. We also simulate the spin-1/2 particle system to illustrate our results. These findings are further validated through numerical illustrations involving a 3D, 5D system, and a spin-1/2 particle system. Full article

(This article belongs to the Special Issue Information Theory in Control Systems, 2nd Edition)

► Show Figures

Figure 1

12 pages, 359 KiB

Open AccessArticle

Statistical Properties of Superpositions of Coherent Phase States with Opposite Arguments

by Miguel Citeli de Freitas and Viktor V. Dodonov

Entropy 2024, 26(11), 977; https://doi.org/10.3390/e26110977 - 15 Nov 2024

Viewed by 180

Abstract

We calculate the second-order moments, the Robertson–Schrödinger uncertainty product, and the Mandel factor for various superpositions of coherent phase states with opposite arguments, comparing the results with similar superpositions of the usual (Klauder–Glauber–Sudarshan) coherent states. We discover that the coordinate variance in the [...] Read more.

We calculate the second-order moments, the Robertson–Schrödinger uncertainty product, and the Mandel factor for various superpositions of coherent phase states with opposite arguments, comparing the results with similar superpositions of the usual (Klauder–Glauber–Sudarshan) coherent states. We discover that the coordinate variance in the analog of even coherent states can show the most strong squeezing effect, close to the maximal possible squeezing for the given mean photon number. On the other hand, the Robertson–Schrödinger (RS) uncertainty product in superpositions of coherent phase states increases much slower (as function of the mean photon number) than in superpositions of the usual coherent states. A nontrivial behavior of the Mandel factor for small mean photon numbers is discovered in superpositions with unequal weights of two components. An exceptional nature of the even and odd superpositions is demonstrated. Full article

(This article belongs to the Special Issue Quantum Probability and Randomness V)

► Show Figures

Figure 1

31 pages, 1865 KiB

Open AccessArticle

Robustness Analysis of Multilayer Infrastructure Networks Based on Incomplete Information Stackelberg Game: Considering Cascading Failures

by Haitao Li, Lixin Ji, Yingle Li and Shuxin Liu

Entropy 2024, 26(11), 976; https://doi.org/10.3390/e26110976 - 14 Nov 2024

Viewed by 278

Abstract

The growing importance of critical infrastructure systems (CIS) makes maintaining their normal operation against deliberate attacks such as terrorism a significant challenge. Combining game theory and complex network theory provides a framework for analyzing CIS robustness in adversarial scenarios. Most existing studies focus [...] Read more.

The growing importance of critical infrastructure systems (CIS) makes maintaining their normal operation against deliberate attacks such as terrorism a significant challenge. Combining game theory and complex network theory provides a framework for analyzing CIS robustness in adversarial scenarios. Most existing studies focus on single-layer networks, while CIS are better modeled as multilayer networks. Research on multilayer network games is limited, lacking methods for constructing incomplete information through link hiding and neglecting the impact of cascading failures. We propose a multilayer network Stackelberg game model with incomplete information considering cascading failures (MSGM-IICF). First, we describe the multilayer network model and define the multilayer node-weighted degree. Then, we present link hiding rules and a cascading failure model. Finally, we construct MSGM-IICF, providing methods for calculating payoff functions from the different perspectives of attackers and defenders. Experiments on synthetic and real-world networks demonstrate that link hiding improves network robustness without considering cascading failures. However, when cascading failures are considered, they become the primary factor determining network robustness. Dynamic capacity allocation enhances network robustness, while changes in dynamic costs make the network more vulnerable. The proposed method provides a new way of analyzing the robustness of diverse CIS, supporting resilient CIS design. Full article

(This article belongs to the Special Issue Robustness and Resilience of Complex Networks)

► Show Figures

Figure 1

13 pages, 668 KiB

Open AccessArticle

Sensitivity of Bayesian Networks to Errors in Their Structure

by Agnieszka Onisko and Marek J. Druzdzel

Entropy 2024, 26(11), 975; https://doi.org/10.3390/e26110975 - 14 Nov 2024

Viewed by 219

Abstract

There is a widespread belief in the Bayesian network (BN) community that while the overall accuracy of the results of BN inference is not sensitive to the precision of parameters, it is sensitive to the structure. We report on the results of a [...] Read more.

There is a widespread belief in the Bayesian network (BN) community that while the overall accuracy of the results of BN inference is not sensitive to the precision of parameters, it is sensitive to the structure. We report on the results of a study focusing on the parameters in a companion paper, while this paper focuses on the BN graphical structure. We present the results of several experiments in which we test the impact of errors in the BN structure on its accuracy in the context of medical diagnostic models. We study the deterioration in model accuracy under structural changes that systematically modify the original gold standard model, notably the node and edge removal and edge reversal. Our results confirm the popular belief that the BN structure is important, and we show that structural errors may lead to a serious deterioration in the diagnostic accuracy. At the same time, most BN models are forgiving to single errors. In light of these results and the results of the companion paper, we recommend that knowledge engineers focus their efforts on obtaining a correct model structure and worry less about the overall precision of parameters. Full article

(This article belongs to the Special Issue Bayesian Network Modelling in Data Sparse Environments)

► Show Figures

Figure 1

Figure 1
The BN models learned from the Breast Cancer data set: using the BSA algorithm (left) and using the ANB algorithm (right). Full article ">Figure 2
The ACC of the Breast Cancer, Cardiotocography, Dermatology, HCV, Hepatitis, Lymphography, Primary Tumor, and SPECT Heart models as a function of the percentage of nodes removed. The colors indicate the ascending, random, and descending order of the cross-entropy between the class node and the removed nodes. Full article ">Figure 3
The ROC curves for the Cardiotocography model when 0%, 20%, and 40% of the nodes have been removed in a descending order. Full article ">Figure 4
The AUC of the Breast Cancer, Cardiotocography, Dermatology, HCV, Hepatitis, Lymphography, Primary Tumor, and SPECT Heart models as a function of the percentage of nodes removed. The colors indicate the ascending, random, and descending order of the cross-entropy between the class node and the removed nodes. Full article ">Figure 5
The AUC of the Breast Cancer, Cardiotocography, Dermatology, HCV, Hepatitis, Lymphography, Primary Tumor, and SPECT Heart models as a function of the percentage of edges removed. The colors indicate the ascending, random, and descending order of the strengths of the removed edges. Full article ">Figure 6
The AUC of the Breast Cancer, Cardiotocography, Dermatology, HCV, Hepatitis, Lymphography, Primary Tumor, and SPECT Heart models as a function of the percentage of edges reversed. The colors indicate the ascending, random, and descending order of the strengths of the reversed edges. Full article ">Figure 6 Cont.
The AUC of the Breast Cancer, Cardiotocography, Dermatology, HCV, Hepatitis, Lymphography, Primary Tumor, and SPECT Heart models as a function of the percentage of edges reversed. The colors indicate the ascending, random, and descending order of the strengths of the reversed edges. Full article ">

27 pages, 12110 KiB

Open AccessArticle

Exploring the Impact of Additive Shortcuts in Neural Networks via Information Bottleneck-like Dynamics: From ResNet to Transformer

by Zhaoyan Lyu and Miguel R. D. Rodrigues

Entropy 2024, 26(11), 974; https://doi.org/10.3390/e26110974 - 14 Nov 2024

Viewed by 277

Abstract

Deep learning has made significant strides, driving advances in areas like computer vision, natural language processing, and autonomous systems. In this paper, we further investigate the implications of the role of additive shortcut connections, focusing on models such as ResNet, Vision Transformers (ViTs), [...] Read more.

Deep learning has made significant strides, driving advances in areas like computer vision, natural language processing, and autonomous systems. In this paper, we further investigate the implications of the role of additive shortcut connections, focusing on models such as ResNet, Vision Transformers (ViTs), and MLP-Mixers, given that they are essential in enabling efficient information flow and mitigating optimization challenges such as vanishing gradients. In particular, capitalizing on our recent information bottleneck approach, we analyze how additive shortcuts influence the fitting and compression phases of training, crucial for generalization. We leverage Z-X and Z-Y measures as practical alternatives to mutual information for observing these dynamics in high-dimensional spaces. Our empirical results demonstrate that models with identity shortcuts (ISs) often skip the initial fitting phase and move directly into the compression phase, while non-identity shortcut (NIS) models follow the conventional two-phase process. Furthermore, we explore how IS models are still able to compress effectively, maintaining their generalization capacity despite bypassing the early fitting stages. These findings offer new insights into the dynamics of shortcut connections in neural networks, contributing to the optimization of modern deep learning architectures. Full article

(This article belongs to the Section Information Theory, Probability and Statistics)

► Show Figures

Figure 1

20 pages, 752 KiB

Open AccessArticle

DUS Topp–Leone-G Family of Distributions: Baseline Extension, Properties, Estimation, Simulation and Useful Applications

by Divine-Favour N. Ekemezie, Kizito E. Anyiam, Mohammed Kayid, Oluwafemi Samson Balogun and Okechukwu J. Obulezi

Entropy 2024, 26(11), 973; https://doi.org/10.3390/e26110973 - 13 Nov 2024

Viewed by 376

Abstract

This study introduces the DUS Topp–Leone family of distributions, a novel extension of the Topp–Leone distribution enhanced by the DUS transformer. We derive the cumulative distribution function (CDF) and probability density function (PDF), demonstrating the distribution’s flexibility in modeling various lifetime phenomena. The [...] Read more.

This study introduces the DUS Topp–Leone family of distributions, a novel extension of the Topp–Leone distribution enhanced by the DUS transformer. We derive the cumulative distribution function (CDF) and probability density function (PDF), demonstrating the distribution’s flexibility in modeling various lifetime phenomena. The DUS-TL exponential distribution was studied as a sub-model, with analytical and graphical evidence revealing that it exhibits a unique unimodal shape, along with fat-tail characteristics, making it suitable for time-to-event data analysis. We evaluate parameter estimation methods, revealing that non-Bayesian approaches, particularly Maximum Likelihood and Least Squares, outperform Bayesian techniques in terms of bias and root mean square error. Additionally, the distribution effectively models datasets with varying skewness and kurtosis values, as illustrated by its application to total factor productivity data across African countries and the mortality rate of people who injected drugs. Overall, the DUS Topp–Leone family represents a significant advancement in statistical modeling, offering robust tools for researchers in diverse fields. Full article

(This article belongs to the Section Information Theory, Probability and Statistics)

► Show Figures

Figure 1

24 pages, 916 KiB

Open AccessArticle

An Instructive CO₂ Adsorption Model for DAC: Wave Solutions and Optimal Processes

by Emily Kay-Leighton and Henning Struchtrup

Entropy 2024, 26(11), 972; https://doi.org/10.3390/e26110972 - 13 Nov 2024

Viewed by 312

Abstract

We present and investigate a simple yet instructive model for the adsorption of CO₂ from air in porous media as used in direct air capture (DAC) processes. Mathematical analysis and non-dimensionalization reveal that the sorbent is characterized by the sorption timescale and [...] Read more.

We present and investigate a simple yet instructive model for the adsorption of CO₂ from air in porous media as used in direct air capture (DAC) processes. Mathematical analysis and non-dimensionalization reveal that the sorbent is characterized by the sorption timescale and capacity, while the adsorption process is effectively wavelike. The systematic evaluation shows that the overall adsorption rate and the recommended charging duration depend only on the wave parameter that is found as the ratio of capacity and dimensionless air flow velocity. Specifically, smaller wave parameters yield a larger overall charging rate, while larger wave parameters reduce the work required to move air through the sorbent. Thus, optimal process conditions must compromise between a large overall adsorption rate and low work requirements. Full article

(This article belongs to the Special Issue The Entropy Production—as Cornerstone in Applied Nonequilibrium Thermodynamics—Dedicated to Professor Signe Kjelstrup on the Occasion of Her 75th Birthday)

► Show Figures

Figure 1

9 pages, 238 KiB

Open AccessArticle

Dirac Equation and Fisher Information

by Asher Yahalom

Entropy 2024, 26(11), 971; https://doi.org/10.3390/e26110971 - 12 Nov 2024

Viewed by 352

Abstract

Previously, it was shown that Schrödinger’s theory can be derived from a potential flow Lagrangian provided a Fisher information term is added. This approach was later expanded to Pauli’s theory of an electron with spin, which required a Clebsch flow Lagrangian with non-zero [...] Read more.

Previously, it was shown that Schrödinger’s theory can be derived from a potential flow Lagrangian provided a Fisher information term is added. This approach was later expanded to Pauli’s theory of an electron with spin, which required a Clebsch flow Lagrangian with non-zero vorticity. Here, we use the recent relativistic flow Lagrangian to represent Dirac’s theory with the addition of a Lorentz invariant Fisher information term as is required by quantum mechanics. Full article

(This article belongs to the Special Issue Applications of Fisher Information in Sciences II)

28 pages, 13144 KiB

Open AccessArticle

Complexity and Variation in Infectious Disease Birth Cohorts: Findings from HIV+ Medicare and Medicaid Beneficiaries, 1999–2020

by Nick Williams

Entropy 2024, 26(11), 970; https://doi.org/10.3390/e26110970 - 12 Nov 2024

Viewed by 380

Abstract

The impact of uncertainty in information systems is difficult to assess, especially when drawing conclusions from human observation records. In this study, we investigate survival variation in a population experiencing infectious disease as a proxy to investigate uncertainty problems. Using Centers for Medicare [...] Read more.

The impact of uncertainty in information systems is difficult to assess, especially when drawing conclusions from human observation records. In this study, we investigate survival variation in a population experiencing infectious disease as a proxy to investigate uncertainty problems. Using Centers for Medicare and Medicaid Services claims, we discovered 1,543,041 HIV+ persons, 363,425 of whom were observed dying from all-cause mortality. Once aggregated by HIV status, year of birth and year of death, Age-Period-Cohort disambiguation and regression models were constructed to produce explanations of variance in survival. We used Age-Period-Cohort as an alternative method to work around under-observed features of uncertainty like infection transmission, receiver host dynamics or comorbidity noise impacting survival variation. We detected ages that have a consistent, disproportionate share of deaths independent of study year or year of birth. Variation in seasonality of mortality appeared stable in regression models; in turn, HIV cases in the United States do not have a survival gain when uncertainty is uncontrolled for. Given the information complexity issues under observed exposure and transmission, studies of infectious diseases should either include robust decedent cases, observe transmission physics or avoid drawing conclusions about survival from human observation records. Full article

(This article belongs to the Special Issue Stability and Flexibility in Dynamic Systems: Novel Research Pathways)

► Show Figures

Figure 1

13 pages, 4232 KiB

Open AccessArticle

Universality of Dynamical Symmetries in Chaotic Maps

by Marcos Acero, Sean Lyons, Andrés Aragoneses and Arjendu K. Pattanayak

Entropy 2024, 26(11), 969; https://doi.org/10.3390/e26110969 - 12 Nov 2024

Viewed by 354

Abstract

Identifying signs of regularity and uncovering dynamical symmetries in complex and chaotic systems is crucial both for practical applications and for enhancing our understanding of complex dynamics. Recent approaches have quantified temporal correlations in time series, revealing hidden, approximate dynamical symmetries that provide [...] Read more.

Identifying signs of regularity and uncovering dynamical symmetries in complex and chaotic systems is crucial both for practical applications and for enhancing our understanding of complex dynamics. Recent approaches have quantified temporal correlations in time series, revealing hidden, approximate dynamical symmetries that provide insight into the systems under study. In this paper, we explore universality patterns in the dynamics of chaotic maps using combinations of complexity quantifiers. We also apply a recently introduced technique that projects dynamical symmetries into a “symmetry space”, providing an intuitive and visual depiction of these symmetries. Our approach unifies and extends previous results and, more importantly, offers a meaningful interpretation of universality by linking it with dynamical symmetries and their transitions. Full article

(This article belongs to the Special Issue Ordinal Patterns-Based Tools and Their Applications)

► Show Figures

Figure 1

35 pages, 2268 KiB

Open AccessArticle

Efficient Search Algorithms for Identifying Synergistic Associations in High-Dimensional Datasets

by Cillian Hourican, Jie Li, Pashupati P. Mishra, Terho Lehtimäki, Binisha H. Mishra, Mika Kähönen, Olli T. Raitakari, Reijo Laaksonen, Liisa Keltikangas-Järvinen, Markus Juonala and Rick Quax

Entropy 2024, 26(11), 968; https://doi.org/10.3390/e26110968 - 11 Nov 2024

Viewed by 400

Abstract

In recent years, there has been a notably increased interest in the study of multivariate interactions and emergent higher-order dependencies. This is particularly evident in the context of identifying synergistic sets, which are defined as combinations of elements whose joint interactions result in [...] Read more.

In recent years, there has been a notably increased interest in the study of multivariate interactions and emergent higher-order dependencies. This is particularly evident in the context of identifying synergistic sets, which are defined as combinations of elements whose joint interactions result in the emergence of information that is not present in any individual subset of those elements. The scalability of frameworks such as partial information decomposition (PID) and those based on multivariate extensions of mutual information, such as O-information, is limited by combinational explosion in the number of sets that must be assessed. In order to address these challenges, we propose a novel approach that utilises stochastic search strategies in order to identify synergistic triplets within datasets. Furthermore, the methodology is extensible to larger sets and various synergy measures. By employing stochastic search, our approach circumvents the constraints of exhaustive enumeration, offering a scalable and efficient means to uncover intricate dependencies. The flexibility of our method is illustrated through its application to two epidemiological datasets: The Young Finns Study and the UK Biobank Nuclear Magnetic Resonance (NMR) data. Additionally, we present a heuristic for reducing the number of synergistic sets to analyse in large datasets by excluding sets with overlapping information. We also illustrate the risks of performing a feature selection before assessing synergistic information in the system. Full article

(This article belongs to the Special Issue Topological Data Analysis Meets Information Theory. New Perspectives for the Analysis of Higher-Order Interactions in Complex Systems)

► Show Figures

Figure 1

13 pages, 464 KiB

Open AccessReview

Entropy of Neuronal Spike Patterns

by Artur Luczak

Entropy 2024, 26(11), 967; https://doi.org/10.3390/e26110967 - 11 Nov 2024

Viewed by 296

Abstract

Neuronal spike patterns are the fundamental units of neural communication in the brain, which is still not fully understood. Entropy measures offer a quantitative framework to assess the variability and information content of these spike patterns. By quantifying the uncertainty and informational content [...] Read more.

Neuronal spike patterns are the fundamental units of neural communication in the brain, which is still not fully understood. Entropy measures offer a quantitative framework to assess the variability and information content of these spike patterns. By quantifying the uncertainty and informational content of neuronal patterns, entropy measures provide insights into neural coding strategies, synaptic plasticity, network dynamics, and cognitive processes. Here, we review basic entropy metrics and then we provide examples of recent advancements in using entropy as a tool to improve our understanding of neuronal processing. It focuses especially on studies on critical dynamics in neural networks and the relation of entropy to predictive coding and cortical communication. We highlight the necessity of expanding entropy measures from single neurons to encompass multi-neuronal activity patterns, as cortical circuits communicate through coordinated spatiotemporal activity patterns, called neuronal packets. We discuss how the sequential and partially stereotypical nature of neuronal packets influences the entropy of cortical communication. Stereotypy reduces entropy by enhancing reliability and predictability in neural signaling, while variability within packets increases entropy, allowing for greater information capacity. This balance between stereotypy and variability supports both robustness and flexibility in cortical information processing. We also review challenges in applying entropy to analyze such spatiotemporal neuronal spike patterns, notably, the “curse of dimensionality” in estimating entropy for high-dimensional neuronal data. Finally, we discuss strategies to overcome these challenges, including dimensionality reduction techniques, advanced entropy estimators, sparse coding schemes, and the integration of machine learning approaches. Thus, this work summarizes the most recent developments on how entropy measures contribute to our understanding of principles underlying neural coding. Full article

(This article belongs to the Section Multidisciplinary Applications)

► Show Figures

Figure 1

7 pages, 220 KiB

Open AccessArticle

An Information-Theoretic Proof of a Hypercontractive Inequality

by Ehud Friedgut

Entropy 2024, 26(11), 966; https://doi.org/10.3390/e26110966 - 11 Nov 2024

Viewed by 296

Abstract

The famous hypercontractive estimate discovered independently by Gross, Bonami and Beckner has had a great impact on combinatorics and theoretical computer science since it was first used in this setting in a seminal paper by Kahn, Kalai and Linial. The usual proofs of [...] Read more.

The famous hypercontractive estimate discovered independently by Gross, Bonami and Beckner has had a great impact on combinatorics and theoretical computer science since it was first used in this setting in a seminal paper by Kahn, Kalai and Linial. The usual proofs of this inequality begin with two-point space, where some elementary calculus is used and then generalised immediately by introducing another dimension using submultiplicativity (Minkowski’s integral inequality). In this paper, we prove this inequality using information theory. We compare the entropy of a pair of correlated vectors in

{0, 1}^{n}

to their separate entropies, analysing them bit by bit (not as a figure of speech, but as the bits are revealed) using the chain rule of entropy. Full article

(This article belongs to the Section Information Theory, Probability and Statistics)

3 pages, 529 KiB

Open AccessCorrection

Correction: Herrera Romero, R.; Bastarrachea-Magnani, M.A. Phase and Amplitude Modes in the Anisotropic Dicke Model with Matter Interactions. Entropy 2024, 26, 574

by Ricardo Herrera Romero and Miguel Angel Bastarrachea-Magnani

Entropy 2024, 26(11), 965; https://doi.org/10.3390/e26110965 - 11 Nov 2024

Viewed by 183

Abstract

The authors wish to make the following correction to this published paper [...] Full article

(This article belongs to the Special Issue Current Trends in Quantum Phase Transitions II)

► Show Figures

Figure 3

25 pages, 1557 KiB

Open AccessArticle

Evidential Analysis: An Alternative to Hypothesis Testing in Normal Linear Models

by Brian Dennis, Mark L. Taper and José M. Ponciano

Entropy 2024, 26(11), 964; https://doi.org/10.3390/e26110964 - 10 Nov 2024

Viewed by 410

Abstract

Statistical hypothesis testing, as formalized by 20th century statisticians and taught in college statistics courses, has been a cornerstone of 100 years of scientific progress. Nevertheless, the methodology is increasingly questioned in many scientific disciplines. We demonstrate in this paper how many of [...] Read more.

Statistical hypothesis testing, as formalized by 20th century statisticians and taught in college statistics courses, has been a cornerstone of 100 years of scientific progress. Nevertheless, the methodology is increasingly questioned in many scientific disciplines. We demonstrate in this paper how many of the worrisome aspects of statistical hypothesis testing can be ameliorated with concepts and methods from evidential analysis. The model family we treat is the familiar normal linear model with fixed effects, embracing multiple regression and analysis of variance, a warhorse of everyday science in labs and field stations. Questions about study design, the applicability of the null hypothesis, the effect size, error probabilities, evidence strength, and model misspecification become more naturally housed in an evidential setting. We provide a completely worked example featuring a two-way analysis of variance. Full article

(This article belongs to the Special Issue Entropy, Statistical Evidence, and Scientific Inference: Evidence Functions in Theory and Applications)

► Show Figures

Figure 1

17 pages, 610 KiB

Open AccessArticle

Sensitivity of Bayesian Networks to Noise in Their Parameters

by Agnieszka Onisko and Marek J. Druzdzel

Entropy 2024, 26(11), 963; https://doi.org/10.3390/e26110963 - 9 Nov 2024

Viewed by 354

Abstract

There is a widely spread belief in the Bayesian network (BN) community that the overall accuracy of results of BN inference is not too sensitive to the precision of their parameters. We present the results of several experiments in which we put this [...] Read more.

There is a widely spread belief in the Bayesian network (BN) community that the overall accuracy of results of BN inference is not too sensitive to the precision of their parameters. We present the results of several experiments in which we put this belief to a test in the context of medical diagnostic models. We study the deterioration of accuracy under random symmetric noise but also biased noise that represents overconfidence and underconfidence of human experts.Our results demonstrate consistently, across all models studied, that while noise leads to deterioration of accuracy, small amounts of noise have minimal effect on the diagnostic accuracy of BN models. Overconfidence, common among human experts, appears to be safer than symmetric noise and much safer than underconfidence in terms of the resulting accuracy. Noise in medical laboratory results and disease nodes as well as in nodes forming the Markov blanket of the disease nodes has the largest effect on accuracy. In light of these results, knowledge engineers should moderately worry about the overall quality of the numerical parameters of BNs and direct their effort where it is most needed, as indicated by sensitivity analysis. Full article

(This article belongs to the Special Issue Bayesian Network Modelling in Data Sparse Environments)

► Show Figures

Figure 1

Figure 1
The Hepar II network. Colors represent the role of each node: yellow are disorder nodes, blue are risk factors, history, and demographic data, and green are symptoms, signs, and laboratory tests. Full article ">Figure 2
Example BN models learned from data: Hepatitis (left) and Breast Cancer (right). The yellow nodes (Class and recurrence) represent class variables. Full article ">Figure 3
Scatterplots of the original (horizontal axis) vs. transformed (vertical axis) probabilities for the Hepar II model. Full article ">Figure 4
Scatterplots of the original (horizontal axis) vs. transformed (vertical axis) probabilities for <math display="inline"><semantics> <mrow> <mi>σ</mi> <mo>=</mo> <mn>0.1</mn> </mrow> </semantics></math> and <math display="inline"><semantics> <mrow> <mi>σ</mi> <mo>=</mo> <mn>0.5</mn> </mrow> </semantics></math>. The top two plots show symmetric noise, the middle two plots show overconfidence, the bottom two plots show underconfidence. Full article ">Figure 5
The average posteriors for the true diagnoses as a function of unbiased (symmetric) noise. Full article ">Figure 6
The posterior probabilities of Hepar II disorders as a function of <math display="inline"><semantics> <mi>σ</mi> </semantics></math> on a single patient case. The lines represent posterior probabilities of the 11 disorders. Full article ">Figure 7
The diagnostic accuracy of the eight models (clock-wise: Hepar II, Lymphography, Primary Tumor, Acute Inflammation, Cardiotocography, Breast Cancer, Hepatitis, and Spect Heart) as a function of the amount of unbiased (symmetric) and biased (overconfidence and underconfidence) noise. Full article ">Figure 8
The diagnostic accuracy of various semantic parts (physical examinations, laboratory results, history, and disorders) of the Hepar II model as a function of the amount of unbiased (symmetric) noise. Full article ">Figure 9
An example of a Bayesian network model with a Markov blanket of the node X. The nodes in green depict the Markov blanket of the node in red. Full article ">Figure 10
The diagnostic accuracy of the four models (clock-wise: Lymphography, Spect Heart, Primary Tumor, and Hepatitis) as a function of the amount of unbiased (symmetric) noise. Full article ">Figure 11
The diagnostic accuracy of the four models (clock-wise: Lymphography, Spect Heart, Primary Tumor, and Hepatitis) as a function of the amount of biased noise (overconfidence). Full article ">Figure 12
The diagnostic accuracy of the four models (clock-wise: Lymphography, Spect Heart, Primary Tumor, and Hepatitis) as a function of the amount of biased (underconfidence). Full article ">

14 pages, 2887 KiB

Open AccessArticle

Machine Learning-Assisted Hartree–Fock Approach for Energy Level Calculations in the Neutral Ytterbium Atom

by Kaichen Ma, Chen Yang, Junyao Zhang, Yunfei Li, Gang Jiang and Junjie Chai

Entropy 2024, 26(11), 962; https://doi.org/10.3390/e26110962 - 8 Nov 2024

Viewed by 410

Abstract

Data-driven machine learning approaches with precise predictive capabilities are proposed to address the long-standing challenges in the calculation of complex many-electron atomic systems, including high computational costs and limited accuracy. In this work, we develop a general workflow for machine learning-assisted atomic structure [...] Read more.

Data-driven machine learning approaches with precise predictive capabilities are proposed to address the long-standing challenges in the calculation of complex many-electron atomic systems, including high computational costs and limited accuracy. In this work, we develop a general workflow for machine learning-assisted atomic structure calculations based on the Cowan code’s Hartree–Fock with relativistic corrections (HFR) theory. The workflow incorporates enhanced ElasticNet and XGBoost algorithms, refined using entropy weight methodology to optimize performance. This semi-empirical framework is applied to calculate and analyze the excited state energy levels of the 4f closed-shell Yb I atom, providing insights into the applicability of different algorithms under various conditions. The reliability and advantages of this innovative approach are demonstrated through comprehensive comparisons with ab initio calculations, experimental data, and other theoretical results. Full article

(This article belongs to the Section Multidisciplinary Applications)

► Show Figures

Figure 1

17 pages, 5532 KiB

Open AccessArticle

Permutation Entropy: An Ordinal Pattern-Based Resilience Indicator for Industrial Equipment

by Christian Salas, Orlando Durán, José Ignacio Vergara and Adolfo Arata

Entropy 2024, 26(11), 961; https://doi.org/10.3390/e26110961 - 8 Nov 2024

Viewed by 359

Abstract

In a highly dynamic and complex environment where risks and uncertainties are inevitable, the ability of a system to quickly recover from disturbances and maintain optimal performance is crucial for ensuring operational continuity and efficiency. In this context, resilience has become an increasingly [...] Read more.

In a highly dynamic and complex environment where risks and uncertainties are inevitable, the ability of a system to quickly recover from disturbances and maintain optimal performance is crucial for ensuring operational continuity and efficiency. In this context, resilience has become an increasingly important topic in the field of engineering and the management of productive systems. However, there is no single quantitative indicator of resilience that allows for the measurement of this characteristic in a productive system. This study proposes the use of permutation entropy of ordinal patterns in time series as an indicator of resilience in industrial equipment and systems. Based on the definition of resilience, the developed method enables precise and efficient assessment of a system’s ability to withstand and recover from disturbances. The methodology includes the identification of ordinal patterns and their analysis through the calculation of a permutation entropy indicator to characterize the dynamics of industrial systems. Case studies are presented and the results are compared with other resilience models existing in the literature, aiming to demonstrate the effectiveness of the proposed approach. The results are promising and highlight a highly applicable and simple indicator for resilience in industrial systems. Full article

(This article belongs to the Special Issue Ordinal Pattern-Based Entropies: New Ideas and Challenges)

► Show Figures

Figure 1

23 pages, 5276 KiB

Open AccessArticle

Generalized Gaussian Distribution Improved Permutation Entropy: A New Measure for Complex Time Series Analysis

by Kun Zheng, Hong-Seng Gan, Jun Kit Chaw, Sze-Hong Teh and Zhe Chen

Entropy 2024, 26(11), 960; https://doi.org/10.3390/e26110960 - 7 Nov 2024

Viewed by 452

Abstract

To enhance the performance of entropy algorithms in analyzing complex time series, generalized Gaussian distribution improved permutation entropy (GGDIPE) and its multiscale variant (MGGDIPE) are proposed in this paper. First, the generalized Gaussian distribution cumulative distribution function is employed for data normalization to [...] Read more.

To enhance the performance of entropy algorithms in analyzing complex time series, generalized Gaussian distribution improved permutation entropy (GGDIPE) and its multiscale variant (MGGDIPE) are proposed in this paper. First, the generalized Gaussian distribution cumulative distribution function is employed for data normalization to enhance the algorithm’s applicability across time series with diverse distributions. The algorithm further processes the normalized data using improved permutation entropy, which maintains both the absolute magnitude and temporal correlations of the signals, overcoming the equal value issue found in traditional permutation entropy (PE). Simulation results indicate that GGDIPE is less sensitive to parameter variations, exhibits strong noise resistance, accurately reveals the dynamic behavior of chaotic systems, and operates significantly faster than PE. Real-world data analysis shows that MGGDIPE provides markedly better separability for RR interval signals, EEG signals, bearing fault signals, and underwater acoustic signals compared to multiscale PE (MPE) and multiscale dispersion entropy (MDE). Notably, in underwater target recognition tasks, MGGDIPE achieves a classification accuracy of 97.5% across four types of acoustic signals, substantially surpassing the performance of MDE (70.5%) and MPE (62.5%). Thus, the proposed method demonstrates exceptional capability in processing complex time series. Full article

(This article belongs to the Special Issue Ordinal Pattern-Based Entropies: New Ideas and Challenges)

► Show Figures

Figure 1

15 pages, 355 KiB

Open AccessArticle

Exact Expressions for Kullback–Leibler Divergence for Univariate Distributions

by Victor Nawa and Saralees Nadarajah

Entropy 2024, 26(11), 959; https://doi.org/10.3390/e26110959 - 7 Nov 2024

Viewed by 465

Abstract

The Kullback–Leibler divergence (KL divergence) is a statistical measure that quantifies the difference between two probability distributions. Specifically, it assesses the amount of information that is lost when one distribution is used to approximate another. This concept is crucial in various fields, including [...] Read more.

The Kullback–Leibler divergence (KL divergence) is a statistical measure that quantifies the difference between two probability distributions. Specifically, it assesses the amount of information that is lost when one distribution is used to approximate another. This concept is crucial in various fields, including information theory, statistics, and machine learning, as it helps in understanding how well a model represents the underlying data. In a recent study by Nawa and Nadarajah, a comprehensive collection of exact expressions for the Kullback–Leibler divergence was derived for both multivariate and matrix-variate distributions. This work is significant as it expands on our existing knowledge of KL divergence by providing precise formulations for over sixty univariate distributions. The authors also ensured the accuracy of these expressions through numerical checks, which adds a layer of validation to their findings. The derived expressions incorporate various special functions, highlighting the mathematical complexity and richness of the topic. This research contributes to a deeper understanding of KL divergence and its applications in statistical analysis and modeling. Full article

(This article belongs to the Section Information Theory, Probability and Statistics)

► Show Figures

Figure 1

14 pages, 321 KiB

Open AccessArticle

On the Negative Result Experiments in Quantum Mechanics

by Kenichi Konishi

Entropy 2024, 26(11), 958; https://doi.org/10.3390/e26110958 - 7 Nov 2024

Viewed by 314

Abstract

We comment on the so-called negative result experiments (also known as null measurements, interaction-free measurements, and so on) in quantum mechanics (QM), in the light of the new general understanding of the quantum-measurement processes, proposed recently. All experiments of this kind (null measurements) [...] Read more.

We comment on the so-called negative result experiments (also known as null measurements, interaction-free measurements, and so on) in quantum mechanics (QM), in the light of the new general understanding of the quantum-measurement processes, proposed recently. All experiments of this kind (null measurements) can be understood as improper measurements with an intentionally biased detector set up, which introduces exclusion or selection of certain events. The prediction on the state of a microscopic system under study based on a null measurement is sometimes dramatically described as “wave-function collapse without any microsystem-detector interactions”. Though certainly correct, such a prediction is just a consequence of the standard QM laws, not different from the situation in the so-called state-preparation procedure. Another closely related concept is the (first-class or) repeatable measurements. The verification of the prediction made by a null measurement requires eventually a standard unbiased measurement involving the microsystem-macroscopic detector interactions, which are nonadiabatic, irreversible processes of signal amplification. Full article

(This article belongs to the Special Issue Quantum Probability and Randomness V)

► Show Figures

Figure 1

19 pages, 4252 KiB

Open AccessArticle

Information Propagation in Hypergraph-Based Social Networks

by Hai-Bing Xiao, Feng Hu, Peng-Yue Li, Yu-Rong Song and Zi-Ke Zhang

Entropy 2024, 26(11), 957; https://doi.org/10.3390/e26110957 - 6 Nov 2024

Viewed by 327

Abstract

Social networks, functioning as core platforms for modern information dissemination, manifest distinctive user clustering behaviors and state transition mechanisms, thereby presenting new challenges to traditional information propagation models. Based on hypergraph theory, this paper augments the traditional SEIR model by introducing a novel [...] Read more.

Social networks, functioning as core platforms for modern information dissemination, manifest distinctive user clustering behaviors and state transition mechanisms, thereby presenting new challenges to traditional information propagation models. Based on hypergraph theory, this paper augments the traditional SEIR model by introducing a novel hypernetwork information dissemination SSEIR model specifically designed for online social networks. This model accurately represents complex, multi-user, high-order interactions. It transforms the traditional single susceptible state (

S

) into active (

S_{a}

) and inactive (

S_{i}

) states. Additionally, it enhances traditional information dissemination mechanisms through reaction process strategies (RP strategies) and formulates refined differential dynamical equations, effectively simulating the dissemination and diffusion processes in online social networks. Employing mean field theory, this paper conducts a comprehensive theoretical derivation of the dissemination mechanisms within the SSEIR model. The effectiveness of the model in various network structures was verified through simulation experiments, and its practicality was further validated by its application on real network datasets. The results show that the SSEIR model excels in data fitting and illustrating the internal mechanisms of information dissemination within hypernetwork structures, further clarifying the dynamic evolutionary patterns of information dissemination in online social hypernetworks. This study not only enriches the theoretical framework of information dissemination but also provides a scientific theoretical foundation for practical applications such as news dissemination, public opinion management, and rumor monitoring in online social networks. Full article

(This article belongs to the Special Issue Spreading Dynamics in Complex Networks)

► Show Figures

Figure 1

15 pages, 8736 KiB

Open AccessArticle

Research on Classification and Identification of Crack Faults in Steam Turbine Blades Based on Supervised Contrastive Learning

by Qinglei Zhang, Laifeng Tang, Jiyun Qin, Jianguo Duan and Ying Zhou

Entropy 2024, 26(11), 956; https://doi.org/10.3390/e26110956 - 6 Nov 2024

Viewed by 376

Abstract

Steam turbine blades may crack, break, or suffer other failures due to high temperatures, high pressures, and high-speed rotation, which seriously threatens the safety and reliability of the equipment. The signal characteristics of different fault types are slightly different, making it difficult to [...] Read more.

Steam turbine blades may crack, break, or suffer other failures due to high temperatures, high pressures, and high-speed rotation, which seriously threatens the safety and reliability of the equipment. The signal characteristics of different fault types are slightly different, making it difficult to accurately classify the faults of rotating blades directly through vibration signals. This method combines a one-dimensional convolutional neural network (1DCNN) and a channel attention mechanism (CAM). 1DCNN can effectively extract local features of time series data, while CAM assigns different weights to each channel to highlight key features. To further enhance the efficacy of feature extraction and classification accuracy, a projection head is introduced in this paper to systematically map all sample features into a normalized space, thereby improving the model’s capacity to distinguish between distinct fault types. Finally, through the optimization of a supervised contrastive learning (SCL) strategy, the model can better capture the subtle differences between different fault types. Experimental results show that the proposed method has an accuracy of 99.61%, 97.48%, and 96.22% in the classification task of multiple crack fault types at three speeds, which is significantly better than Multilayer Perceptron (MLP), Residual Network (ResNet), Momentum Contrast (MoCo), and Transformer methods. Full article

(This article belongs to the Section Multidisciplinary Applications)

► Show Figures

Figure 1

Figure 1
System framework diagram (The figure illustrates a fault diagnosis framework based on vibration signal data. The experiment collects vibration signals of blade crack faults, constructs a fault dataset, and adds Gaussian noise. The 1D CNN is combined with CAM to extract fault features. A projection head is introduced to map all sample features into a normalized space, thereby enhancing the model’s ability to distinguish between different fault types. Additionally, contrast loss and cross-entropy loss are calculated through supervised contrast learning to complete the fault classification.). Full article ">Figure 2
Vibration signals before and after adding Gaussian noise. ((a,c,e,g,i) represent the “normal” without adding Gaussian noise, the fault signal of “crack1”, the fault signal of “crack2”, the fault signal of “crack3”, and the fault signal of “fracture”, respectively; (b,d,f,h,j) represent the above five signals after adding Gaussian noise, respectively.). Full article ">Figure 3
Dynamic test bench of rotor system with integral shroud blade. Full article ">Figure 4
(a–c) represent the accuracy of MLP, ResNet, MoCo, Transformer, and our proposed method at 1400 r/min, 1800 r/min, and 2200 r/min, respectively. Full article ">Figure 5
(a–c) represent the confusion matrix results at 1400 r/min, 1800 r/min, and 2200 r/min. The values on the diagonal represent the number of samples predicted correctly, and the values on the off-diagonal represent the number of samples predicted incorrectly. Full article ">Figure 6
t-SNE feature visualization: (a) 1400 r/min, (b) 1800 r/min, (c) 2200 r/min. Full article ">

13 pages, 577 KiB

Open AccessArticle

Identifying Key Nodes for the Influence Spread Using a Machine Learning Approach

by Mateusz Stolarski, Adam Piróg and Piotr Bródka

Entropy 2024, 26(11), 955; https://doi.org/10.3390/e26110955 - 6 Nov 2024

Viewed by 360

Abstract

The identification of key nodes in complex networks is an important topic in many network science areas. It is vital to a variety of real-world applications, including viral marketing, epidemic spreading and influence maximization. In recent years, machine learning algorithms have proven to [...] Read more.

The identification of key nodes in complex networks is an important topic in many network science areas. It is vital to a variety of real-world applications, including viral marketing, epidemic spreading and influence maximization. In recent years, machine learning algorithms have proven to outperform the conventional, centrality-based methods in accuracy and consistency, but this approach still requires further refinement. What information about the influencers can be extracted from the network? How can we precisely obtain the labels required for training? Can these models generalize well? In this paper, we answer these questions by presenting an enhanced machine learning-based framework for the influence spread problem. We focus on identifying key nodes for the Independent Cascade model, which is a popular reference method. Our main contribution is an improved process of obtaining the labels required for training by introducing “Smart Bins” and proving their advantage over known methods. Next, we show that our methodology allows ML models to not only predict the influence of a given node, but to also determine other characteristics of the spreading process—which is another novelty to the relevant literature. Finally, we extensively test our framework and its ability to generalize beyond complex networks of different types and sizes, gaining important insight into the properties of these methods. Full article

(This article belongs to the Section Multidisciplinary Applications)

► Show Figures

Figure 1

19 pages, 2580 KiB

Open AccessArticle

A Hybrid Quantum-Classical Model for Stock Price Prediction Using Quantum-Enhanced Long Short-Term Memory

by Kimleang Kea, Dongmin Kim, Chansreynich Huot, Tae-Kyung Kim and Youngsun Han

Entropy 2024, 26(11), 954; https://doi.org/10.3390/e26110954 - 6 Nov 2024

Viewed by 428

Abstract

The stock markets have become a popular topic within machine learning (ML) communities, with one particular application being stock price prediction. However, accurately predicting the stock market is a challenging task due to the various factors within financial markets. With the introduction of [...] Read more.

The stock markets have become a popular topic within machine learning (ML) communities, with one particular application being stock price prediction. However, accurately predicting the stock market is a challenging task due to the various factors within financial markets. With the introduction of ML, prediction techniques have become more efficient but computationally demanding for classical computers. Given the rise of quantum computing (QC), which holds great promise for being exponentially faster than current classical computers, it is natural to explore ML within the QC domain. In this study, we leverage a hybrid quantum-classical ML approach to predict a company’s stock price. We integrate classical long short-term memory (LSTM) with QC, resulting in a new variant called QLSTM. We initially validate the proposed QLSTM model by leveraging an IBM quantum simulator running on a classical computer, after which we conduct predictions using an IBM real quantum computer. Thereafter, we evaluate the performance of our model using the root mean square error (RMSE) and prediction accuracy. Additionally, we perform a comparative analysis, evaluating the prediction performance of the QLSTM model against several other classical models. Further, we explore the impacts of hyperparameters on the QLSTM model to determine the best configuration. Our experimental results demonstrate that while the classical LSTM model achieved an RMSE of 0.0693 and a prediction accuracy of 0.8815, the QLSTM model exhibited superior performance, achieving values of 0.0602 and 0.9736, respectively. Furthermore, the QLSTM outperformed other classical models in both metrics. Full article

(This article belongs to the Special Issue The Future of Quantum Machine Learning and Quantum AI)

► Show Figures

Figure 1

71 pages, 3052 KiB

Open AccessPerspective

The Algorithmic Agent Perspective and Computational Neuropsychiatry: From Etiology to Advanced Therapy in Major Depressive Disorder

by Giulio Ruffini, Francesca Castaldo, Edmundo Lopez-Sola, Roser Sanchez-Todo and Jakub Vohryzek

Entropy 2024, 26(11), 953; https://doi.org/10.3390/e26110953 - 6 Nov 2024

Viewed by 445

Abstract

Major Depressive Disorder (MDD) is a complex, heterogeneous condition affecting millions worldwide. Computational neuropsychiatry offers potential breakthroughs through the mechanistic modeling of this disorder. Using the Kolmogorov theory (KT) of consciousness, we developed a foundational model where algorithmic agents interact with the world [...] Read more.

Major Depressive Disorder (MDD) is a complex, heterogeneous condition affecting millions worldwide. Computational neuropsychiatry offers potential breakthroughs through the mechanistic modeling of this disorder. Using the Kolmogorov theory (KT) of consciousness, we developed a foundational model where algorithmic agents interact with the world to maximize an Objective Function evaluating affective valence. Depression, defined in this context by a state of persistently low valence, may arise from various factors—including inaccurate world models (cognitive biases), a dysfunctional Objective Function (anhedonia, anxiety), deficient planning (executive deficits), or unfavorable environments. Integrating algorithmic, dynamical systems, and neurobiological concepts, we map the agent model to brain circuits and functional networks, framing potential etiological routes and linking with depression biotypes. Finally, we explore how brain stimulation, psychotherapy, and plasticity-enhancing compounds such as psychedelics can synergistically repair neural circuits and optimize therapies using personalized computational models. Full article

(This article belongs to the Special Issue From Functional Imaging to Free Energy—Dedicated to Professor Karl Friston on the Occasion of His 65th Birthday)

► Show Figures

Figure 1

Figure 1
Conceptual Roadmap—From agent theory to treatment. The algorithmic agent framework provides conceptual links between first-person experience and information processing and guides the search for the causes of MDD (see <a href="#sec2-entropy-26-00953" class="html-sec">Section 2</a>). The translational framework represents the various theoretical functional modules from the perspective of circuits and dynamics with special emphasis on the clinical biotypes pertaining to MDD. Based on brain circuits and individualized biotypes, different MDD interventions are proposed for treatment. Full article ">Figure 2
(a) Generic agent model. The agent interacts dynamically with its environment and is involved in continuously exchanging data with the external world. The modeling engine runs the model and makes predictions of future data (both from external interfaces and the agent’s own actions). Then, the prediction error (from the Comparator) is evaluated to update the model. The Updater receives prediction errors from the Comparator to improve the model. The Simulator is a shared infrastructure used to run simulations for planning or valence evaluation. The Planning Engine runs counterfactual simulations and selects plans for the next actions (agent outputs). A key agent element is the Objective Function, which the agent aims to optimize through a choice of afferent actions selected by the Planning Engine. (b) MDD agent. Non-exclusive dysfunction across the main agent components (b1–b3) or hostile world inputs (b4) can result in sustained low output values from the Objective Function (low valence, red arrow). Identifying those can help us identify the routes that lead the agent to a depressed state. Full article ">Figure 3
From the agent model to brain circuits, functional networks, and biotypes. To highlight the goals of the proposed framework, we provide a tentative map or agent modules to brain regions, circuits, and candidate biotypes based on the current literature. Proposed mapping of agent model and agent elements (a) to high-level brain circuits (b)—Modeling Engine in pink, Planning Engine in blue, and Objective Function (valence evaluation) in yellow. The high-level circuital model maps into structural/anatomical circuits (c) and, finally, into observed functional networks that can be derived from fMRI (d). Features from these networks can be used as biomarkers for the definition of patient clusters or FN biotypes (e) [<a href="#B10-entropy-26-00953" class="html-bibr">10</a>,<a href="#B11-entropy-26-00953" class="html-bibr">11</a>,<a href="#B12-entropy-26-00953" class="html-bibr">12</a>,<a href="#B65-entropy-26-00953" class="html-bibr">65</a>,<a href="#B100-entropy-26-00953" class="html-bibr">100</a>]. The characteristics of each biotype (alterations in the activity of regions or their connectivity) will reflect alterations of the main agent modules involved. As an example, we display a particular FN biotype (<math display="inline"><semantics> <mrow> <msub> <mrow> <mi mathvariant="normal">N</mi> <mi>S</mi> </mrow> <mrow> <mi>A</mi> <mo>+</mo> </mrow> </msub> <msub> <mi mathvariant="normal">P</mi> <mrow> <mi>A</mi> <mo>+</mo> </mrow> </msub> </mrow> </semantics></math> in [<a href="#B12-entropy-26-00953" class="html-bibr">12</a>]), with altered activity in the Objective Function (AMY, BG, and ACC). Most FN biotypes present alterations in more than one agent module. Full article ">Figure 4
The agent and dynamical landscape frameworks provide a theoretical basis for the development and interpretation of data analysis methods and mechanistic computational models of the brain. This can be employed for a model-based design of therapeutic interventions such as pharmacology, brain stimulation, or even psychotherapy, as well as their combination. In the figure, we display the process of data assimilation from a group of reference healthy subjects/controls (HCs) and from MDD patients of a particular subtype. The resulting sets of models can guide the design of an intervention for the normalization of the dynamics and connectivity profile of the patients. The treatment can be designed at the group [<a href="#B289-entropy-26-00953" class="html-bibr">289</a>] or individual level. Full article ">Figure A1
KT and AIF agent models and etiology of MDD. Schematic view of the agent models in KT and FEP/AIF. (a) In KT, the Objective Function is explicit, while in AIF, it is implicit in the loss function (free energy). (b) In AIF, the Planning Engine seeks to minimize free energy or surprise (negative valence). All the agent elements, in either case, can be affected and lead to depression in KT, i.e., persistent low valence: (i) Modeling Engine creates models leading to low valence; (ii) Objective Function returns low valence regardless of the models and displays no plasticity to “recalibrate”; (iii) Planning Engine delivers ineffective or detrimental plans for valence; and (iv) world provides inputs with an implicit stasis threat that cascades down to low valence. Further details: The agent interacts dynamically with its environment and is involved in a continuous exchange of data with the external world. The Modeling Engine runs the model and makes predictions of future data (both from external interfaces as well as the agent’s own actions) and then evaluates the prediction error (from the Comparator C) to update the model. The Simulator (not shown for simplicity) is a shared infrastructure used to run simulations for planning or valence evaluation. The Planning Engine (P) runs counterfactual simulations and selects plans for the next actions (agent outputs). The Updater receives as inputs prediction errors from the Comparator to improve the model. The other agent element is the Objective Function (which may also be plastic), which the agent aims to optimize through a choice of afferent actions selected by the Planning Engine. Full article ">Figure A2
The hierarchical agent: Agent diagram displaying hierarchical processing of inputs (as in the predictive coding framework) and with a segregated evaluation of valence, which is also hierarchical, since it relies on the same hierarchical simulation engine. Full article ">

Journal Menu

Journal Browser

Entropy, Volume 26, Issue 11 (November 2024) – 94 articles

Further Information

Guidelines

MDPI Initiatives

Follow MDPI