The Liang-Kleeman Information Flow: Theory and Applications
<p>Illustration of the Frobenius-Perron operator <math display="inline"> <mi mathvariant="script">P</mi> </math>, which takes <math display="inline"> <mrow> <mi>ρ</mi> <mo>(</mo> <mi mathvariant="bold-italic">x</mi> <mo>)</mo> </mrow> </math> to <math display="inline"> <mrow> <mi mathvariant="script">P</mi> <mi>ρ</mi> <mo>(</mo> <mi mathvariant="bold-italic">x</mi> <mo>)</mo> </mrow> </math> as Φ takes <math display="inline"> <mi mathvariant="bold-italic">x</mi> </math> to <math display="inline"> <mrow> <mo>Φ</mo> <mi mathvariant="bold-italic">x</mi> </mrow> </math>.</p> "> Figure 2
<p>Illustration of the unidirectional information flow within the baker transformation.</p> "> Figure 3
<p>A trajectory of the canonical H<math display="inline"> <mover accent="true"> <mi mathvariant="normal">e</mi> <mo>´</mo> </mover> </math>non map (<math display="inline"> <mrow> <mi>a</mi> <mo>=</mo> <mn>1.4</mn> </mrow> </math>, <math display="inline"> <mrow> <mi>b</mi> <mo>=</mo> <mn>0.3</mn> </mrow> </math>) starting at <math display="inline"> <mrow> <mrow> <mo>(</mo> <msub> <mi>x</mi> <mn>1</mn> </msub> <mo>,</mo> <msub> <mi>x</mi> <mn>2</mn> </msub> <mo>)</mo> </mrow> <mo>=</mo> <mrow> <mo>(</mo> <mn>1</mn> <mo>,</mo> <mn>0</mn> <mo>)</mo> </mrow> </mrow> </math>.</p> "> Figure 4
<p>The invariant attractor of the truncated Burgers–Hopf system (73)–(76). Shown here is the trajectory segment for <math display="inline"> <mrow> <mn>2</mn> <mo>≤</mo> <mi>t</mi> <mo>≤</mo> <mn>20</mn> </mrow> </math> starting at <math display="inline"> <mrow> <mo>(</mo> <mn>40</mn> <mo>,</mo> <mn>40</mn> <mo>,</mo> <mn>40</mn> <mo>,</mo> <mn>40</mn> <mo>)</mo> </mrow> </math>. (a) and (b) are the 3-dimensional projections onto the subspaces <math display="inline"> <msub> <mi>x</mi> <mn>1</mn> </msub> </math>-<math display="inline"> <msub> <mi>x</mi> <mn>2</mn> </msub> </math>-<math display="inline"> <msub> <mi>x</mi> <mn>3</mn> </msub> </math> and <math display="inline"> <msub> <mi>x</mi> <mn>2</mn> </msub> </math>-<math display="inline"> <msub> <mi>x</mi> <mn>3</mn> </msub> </math>-<math display="inline"> <msub> <mi>x</mi> <mn>4</mn> </msub> </math>, respectively.</p> "> Figure 5
<p>Information flows within the 4D truncated Burgers-Hopf system. The series prior to <math display="inline"> <mrow> <mi>t</mi> <mo>=</mo> <mn>2</mn> </mrow> </math> are not shown because some trajectories have not entered the attractor by that time.</p> "> Figure 6
<p>A solution of Equation (78), the model examined in [<a href="#B54-entropy-15-00327" class="html-bibr">54</a>], with <math display="inline"> <mrow> <msub> <mi>a</mi> <mn>21</mn> </msub> <mo>=</mo> <mn>0</mn> </mrow> </math> and initial conditions as shown in the text: (<b>a</b>) <math display="inline"> <mi mathvariant="bold">μ</mi> </math>; (<b>b</b>) Σ; and (<b>c</b>) a sample path starting from (1,2).</p> "> Figure 7
<p>The computed rates of information flow for the system (77): (a) <math display="inline"> <msub> <mi>T</mi> <mrow> <mn>2</mn> <mo>→</mo> <mn>1</mn> </mrow> </msub> </math>, (b) <math display="inline"> <msub> <mi>T</mi> <mrow> <mn>1</mn> <mo>→</mo> <mn>2</mn> </mrow> </msub> </math>.</p> ">
Abstract
:1. Introduction
2. Mathematical Formalism
2.1. Theoretical Framework
2.2. Toward a Rigorous Formalism—A Heuristic Argument
2.3. Mathematical Formalism
3. Discrete Systems
3.1. Frobenius-Perron Operator
3.2. Information Flow
3.3. Properties
4. Continuous Systems
- Discretize the continuous system in time on , and construct a mapping Φ to take to ;
- Freeze in Φ throughout to obtain a modified mapping ;
- Compute the marginal entropy change as Φ steers the system from t to ;
- Derive the marginal entropy change as steers the modified system from t to ;
- Take the limit
4.1. Discretization of the Continuous System
4.2. Information Flow
4.3. Properties
5. Stochastic Systems
6. Applications
6.1. Baker Transformation
6.2. Hnon Map
6.3. Truncated Burgers–Hopf System
- Initialize the joint density of with some distribution ; make random draws according to to form an ensemble. The ensemble should be large enough to resolve adequately the sample space.
- Discretize the sample space into “bins.”
- Do ensemble prediction for the system (73)–(74).
- At each step, estimate the probability density function ρ by counting the bins.
- Plug the estimated ρ back to Equation (39) to compute the rates of information flow at that step.
6.4. Langevin Equation
7. Summary
Acknowledgments
References
- Baptista, M.S.; Garcia, S.P.; Dana, S.K.; Kurths, J. Transmission of information and synchronization in a pair of coupled chaotic circuits: An experimental overview. Eur. Phys. J.-Spec. Top. 2008, 165, 119–128. [Google Scholar] [CrossRef]
- Baptista, M.D.S.; Kakmeni, F.M.; Grebogi, C. Combined effect of chemical and electrical synapses in Hindmarsh-Rose neural networks on synchronization and the rate of information. Phys. Rev. E 2010, 82, 036203. [Google Scholar] [CrossRef] [PubMed]
- Bear, M.F.; Connors, B.W.; Paradiso, M.A. Neuroscience: Exploring the Brain, 3rd ed.; Lippincott Williams & Wilkins: Baltimore, MD, USA, 2007; p. 857. [Google Scholar]
- Vakorin, V.A.; MiAiA, B.; Krakovska, O.; McIntosh, A.R. Empirical and theoretical aspects of generation and transfer of information in a neuromagnetic source network. Front. Syst. Neurosci. 2011, 5, 96. [Google Scholar] [CrossRef] [PubMed]
- Ay, N.; Polani, D. Information flows in causal networks. Advs. Complex Syst. 2008, 11. [Google Scholar] [CrossRef]
- Peruani, F.; Tabourier, L. Directedness of information flow in mobile phone communication networks. PLoS One 2011, 6, e28860. [Google Scholar] [CrossRef] [PubMed]
- Sommerlade, L.; Amtage, F.; Lapp, O.; Hellwig, B.; Licking, C.H.; Timmer, J.; Schelter, B. On the estimation of the direction of information flow in networks of dynamical systems. J. Neurosci. Methods 2011, 196, 182–189. [Google Scholar] [CrossRef] [PubMed]
- Donner, R.; Barbosa, S.; Kurths, J.; Marwan, N. Understanding the earth as a complex system-recent advances in data analysis and modelling in earth sciences. Eur. Phys. J. 2009, 174, 1–9. [Google Scholar] [CrossRef]
- Kleeman, R. Information flow in ensemble weather prediction. J. Atmos. Sci. 2007, 64, 1005–1016. [Google Scholar] [CrossRef]
- Materassi, M.; Ciraolo, L.; Consolini, G.; Smith, N. Predictive space weather: An information theory approach. Adv. Space Res. 2011, 47, 877–885. [Google Scholar] [CrossRef]
- Tribbia, J.J. Waves, Information and Local Predictability. In Proceedings of the Workshop on Mathematical Issues and Challenges in Data Assimilation for Geophysical Systems: Interdisciplinary Perspectives, IPAM, UCLA, 22–25 February 2005.
- Chen, C.R.; Lung, P.P.; Tay, N.S.P. Information flow between the stock and option markets: Where do informed traders trade? Rev. Financ. Econ. 2005, 14, 1–23. [Google Scholar] [CrossRef]
- Lee, S.S. Jumps and information flow in financial markets. Rev. Financ. Stud. 2012, 25, 439–479. [Google Scholar] [CrossRef]
- Sommerlade, L.; Eichler, M.; Jachan, M.; Henschel, K.; Timmer, J.; Schelter, B. Estimating causal dependencies in networks of nonlinear stochastic dynamical systems. Phys. Rev. E 2009, 80, 051128. [Google Scholar] [CrossRef] [PubMed]
- Zhao, K.; Karsai, M.; Bianconi, G. Entropy of dynamical social networks. PLoS One 2011. [Google Scholar] [CrossRef] [PubMed]
- Cane, M.A. The evolution of El Niño, past and future. Earth Planet. Sci. Lett. 2004, 164, 1–10. [Google Scholar] [CrossRef]
- Jin, F.-F. An equatorial ocean recharge paradigm for ENSO. Part I: conceptual model. J. Atmos. Sci. 1997, 54, 811–829. [Google Scholar] [CrossRef]
- Philander, S.G. El Niño, La Niña, and the Southern Oscillation; Academic Press: San Diego, CA, USA, 1990. [Google Scholar]
- Ghil, M.; Chekroun, M.D.; Simonnet, E. Climate dynamics and fluid mechanics: Natural variability and related uncertainties. Physica D 2008, 237, 2111–2126. [Google Scholar] [CrossRef]
- Mu, M.; Xu, H.; Duan, W. A kind of initial errors related to “spring predictability barrier” for El Niño events in Zebiak-Cane model. Geophys. Res. Lett. 2007, 34, L03709. [Google Scholar] [CrossRef]
- Zebiak, S.E.; Cane, M.A. A model El Niño-Southern Oscillation. Mon. Wea. Rev. 1987, 115, 2262–2278. [Google Scholar] [CrossRef]
- Chen, D.; Cane, M.A. El Niño prediction and predictability. J. Comput. Phys. 2008, 227, 3625–3640. [Google Scholar] [CrossRef]
- Mayhew, S.; Sarin, A.; Shastri, K. The allocation of informed trading across related markets: An analysis of the impact of changes in equity-option margin requirements. J. Financ. 1995, 50, 1635–1654. [Google Scholar] [CrossRef]
- Goldenfield, N.; Woese, C. Life is physics: Evolution as a collective phenomenon far from equilibrium. Ann. Rev. Condens. Matt. Phys. 2011, 2, 375–399. [Google Scholar] [CrossRef]
- K¨ppers, B. Information and the Origin of Life; MIT Press: Cambridge, UK, 1990. [Google Scholar]
- Murray, J.D. Mathematical Biology; Springer-Verlag: Berlin, Germany, 2000. [Google Scholar]
- Allahverdyan, A.E.; Janzing, D.; Mahler, G. Thermodynamic efficiency of information and heat flow. J. Stat. Mech. 2009, PO9011. [Google Scholar] [CrossRef]
- Crutchfield, J.P.; Shalizi, C.R. Thermodynamic depth of causal states: Objective complexity via minimal representation. Phys. Rev. E 1999, 59, 275–283. [Google Scholar] [CrossRef]
- Davies, P.C.W. The Physics of Downward Causation. In The Re-emergence of Emergence; Clayton, P., Davies, P.C.W., Eds.; Oxford University Press: Oxford, UK, 2006; pp. 35–52. [Google Scholar]
- Ellis, G.F.R. Top-down causation and emergence: Some comments on mechanisms. J. R. Soc. Interface 2012, 2, 126–140. [Google Scholar] [CrossRef] [PubMed]
- Okasha, S. Emergence, hierarchy and top-down causation in evolutionary biology. J. R. Soc. Interface 2012, 2, 49–54. [Google Scholar] [CrossRef] [PubMed]
- Walker, S.I.; Cisneros, L.; Davies, P.C.W. Evolutionary transitions and top-down causation. arXiv:1207.4808v1 [nlin.AO], 2012. [Google Scholar]
- Wu, B.; Zhou, D.; Fu, F.; Luo, Q.; Wang, L.; Traulsen, A. Evolution of cooperation on stochastic dynamical networks. PLoS One 2010, 5, e11187. [Google Scholar] [CrossRef] [PubMed]
- Pope, S. Turbulent Flows, 8th ed.; Cambridge University Press: Cambridge, UK, 2011. [Google Scholar]
- Faes, L.; Nollo, G.; Erla, S.; Papadelis, C.; Braun, C.; Porta, A. Detecting Nonlinear Causal Interactions between Dynamical Systems by Non-uniform Embedding of Multiple Time Series. In Proceedings of the Engineering in Medicine and Biology Society, Buenos Aires, Argentina, 31 August–4 September 2010; pp. 102–105.
- Kantz, H.; Shreiber, T. Nonlinear Time Series Analysis; Cambridge University Press: Cambridge, UK, 2004. [Google Scholar]
- Schindler-Hlavackova, K.; Palus, M.; Vejmelka, M.; Bhattacharya, J. Causality detection based on information-theoretic approach in time series analysis. Phys. Rep. 2007, 441, 1–46. [Google Scholar] [CrossRef]
- Granger, C. Investigating causal relations by econometric models and cross-spectral methods. Econometrica 1969, 37, 424–438. [Google Scholar] [CrossRef]
- McWilliams, J.C. The emergence of isolated, coherent vortices in turbulence flows. J. Fluid Mech. 1984, 146, 21–43. [Google Scholar] [CrossRef]
- Salmon, R. Lectures on Geophysical Fluid Dynamics; Oxford University Press: Oxford, UK, 1998; p. 378. [Google Scholar]
- Bar-Yam, Y. Dynamics of Complex Systems; Addison-Welsley Press: Reading, MA, USA, 1997; p. 864. [Google Scholar]
- Crutchfield, J.P. The calculi of emergence: computation, dynamics, and induction induction. “Special issue on the Proceedings of the Oji International Seminar: Complex Systems-From Complex Dynamics to Artifical Reality”. Physica D 1994, 75, 11–54. [Google Scholar] [CrossRef]
- Goldstein, J. Emergence as a construct: History and issues. Emerg. Complex. Org. 1999, 1, 49–72. [Google Scholar] [CrossRef]
- Corning, P.A. The re-emergence of emergence: A venerable concept in search of a theory. Complexity 2002, 7, 18–30. [Google Scholar] [CrossRef]
- Vastano, J.A.; Swinney, H.L. Information transport in sptiotemporal systems. Phys. Rev. Lett. 1988, 60, 1773–1776. [Google Scholar] [CrossRef] [PubMed]
- Kaiser, A.; Schreiber, T. Information transfer in continuous processes. Physica D 2002, 166, 43–62. [Google Scholar] [CrossRef]
- Schreiber, T. Measuring information transfer. Phys. Rev. Lett. 2000, 85, 461. [Google Scholar] [CrossRef] [PubMed]
- Liang, X.S.; Kleeman, R. A rigorous formalism of information transfer between dynamical system components. I. Discrete mapping. Physica D 2007, 231, 1–9. [Google Scholar] [CrossRef]
- Liang, X.S.; Kleeman, R. Information transfer between dynamical system components. Phys. Rev. Lett. 2005, 95, 244101. [Google Scholar] [CrossRef] [PubMed]
- Lasota, A.; Mackey, M.C. Chaos, Fractals, and Noise: Stochastic Aspects of Dynamics; Springer: New York, NY, USA, 1994. [Google Scholar]
- Liang, X.S.; Kleeman, R. A rigorous formalism of information transfer between dynamical system components. II. Continuous flow. Physica D 2007, 227, 173–182. [Google Scholar] [CrossRef]
- Liang, X.S. Uncertainty generation in deterministic fluid flows: Theory and applications with an atmospheric stability model. Dyn. Atmos. Oceans 2011, 52, 51–79. [Google Scholar] [CrossRef]
- Majda, A.J.; Harlim, J. Information flow between subspaces of complex dynamical systems. Proc. Natl. Acad. Sci. USA 2007, 104, 9558–9563. [Google Scholar] [CrossRef]
- Liang, X.S. Information flow within stochastic dynamical systems. Phys. Rev. E 2008, 78, 031113. [Google Scholar] [CrossRef] [PubMed]
- Majda, A.J.; Timofeyev, I. Remarkable statistical behavior for truncated Burgers-Hopf dynamics. Proc. Natl. Acad. Sci. USA 2000, 97, 12413–12417. [Google Scholar] [CrossRef] [PubMed]
- Gardiner, C.W. Handbook of Stochastic Methods for Physics, Chemistry, and the Natural Sciences; Springer-Verlag: Berlin/Heidelberg, Germany, 1985. [Google Scholar]
- Liang, X.S.; Anderson, D.G.M.A. Multiscale window transform. SIAM J. Multiscale Model. Simul. 2007, 6, 437–467. [Google Scholar] [CrossRef]
- Liang, X.S.; Robinson, A.R. Multiscale processes and nonlinear dynamics of the circulation and upwelling events off Monterey Bay. J. Phys. Oceanogr. 2009, 39, 290–313. [Google Scholar] [CrossRef]
- Kleeman, R. Measuring dynamical prediction utility using relative entropy. J. Atmos. Sci. 2002, 59, 2057–2072. [Google Scholar] [CrossRef]
- Cover, T.M.; Thomas, J.A. Elements of Information Theory; Wiley: New York, NY, USA, 1991. [Google Scholar]
- Ao, P. Emerging of stochastic dynamical equalities and steady state thermodynamics from Darwinian dynamics. Commun. Theor. Phys. 2008, 49, 1073–1090. [Google Scholar] [CrossRef] [PubMed]
- Tang, Y.; Deng, Z.; Zhou, X.; Cheng, Y.; Chen, D. Interdecadal variation of ENSO predictability in multiple models. J. Clim. 2008, 21, 4811–4832. [Google Scholar] [CrossRef]
© 2013 by the authors; licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution license (http://creativecommons.org/licenses/by/3.0/).
Share and Cite
Liang, X.S. The Liang-Kleeman Information Flow: Theory and Applications. Entropy 2013, 15, 327-360. https://doi.org/10.3390/e15010327
Liang XS. The Liang-Kleeman Information Flow: Theory and Applications. Entropy. 2013; 15(1):327-360. https://doi.org/10.3390/e15010327
Chicago/Turabian StyleLiang, X. San. 2013. "The Liang-Kleeman Information Flow: Theory and Applications" Entropy 15, no. 1: 327-360. https://doi.org/10.3390/e15010327