[go: up one dir, main page]
Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

Information-limiting correlations

Nature Neuroscience volume 17, pages 1410–1417 (2014)Cite this article

Subjects

Abstract

Computational strategies used by the brain strongly depend on the amount of information that can be stored in population activity, which in turn strongly depends on the pattern of noise correlations. In vivo, noise correlations tend to be positive and proportional to the similarity in tuning properties. Such correlations are thought to limit information, which has led to the suggestion that decorrelation increases information. In contrast, we found, analytically and numerically, that decorrelation does not imply an increase in information. Instead, the only information-limiting correlations are what we refer to as differential correlations: correlations proportional to the product of the derivatives of the tuning curves. Unfortunately, differential correlations are likely to be very small and buried under correlations that do not limit information, making them particularly difficult to detect. We found, however, that the effect of differential correlations on information can be detected with relatively simple decoders.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on SpringerLink
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Figure 1: The effect of correlations on a population code with translation invariant tuning curves.
Figure 2: Decorrelation does not necessarily increase information.
Figure 3: Differential correlations induced by a shifting hill.
Figure 4: Shared connections do not necessarily induce information limiting correlations.
Figure 5: Differential correlations.
Figure 6: Small differential correlations can have a large effect on information.
Figure 7: Estimating information from neuronal populations with differential correlations (Supplementary Modeling).

Similar content being viewed by others

References

  1. Tolhurst, D.J., Movshon, J.A. & Dean, A.D. The statistical reliability of signals in single neurons in cat and monkey visual cortex. Vision Res. 23, 775–785 (1983).

    Article  CAS  Google Scholar 

  2. Averbeck, B.B., Latham, P.E. & Pouget, A. Neural correlations, population coding and computation. Nat. Rev. Neurosci. 7, 358–366 (2006).

    Article  CAS  Google Scholar 

  3. Zohary, E., Shadlen, M.N. & Newsome, W.T. Correlated neuronal discharge rate and its implication for psychophysical performance. Nature 370, 140–143 (1994).

    Article  CAS  Google Scholar 

  4. Maynard, E.M. et al. Neuronal interactions improve cortical population coding of movement direction. J. Neurosci. 19, 8083–8093 (1999).

    Article  CAS  Google Scholar 

  5. Averbeck, B.B. & Lee, D. Neural noise and movement-related codes in the macaque supplementary motor area. J. Neurosci. 23, 7630–7641 (2003).

    Article  CAS  Google Scholar 

  6. Gu, Y. et al. Perceptual learning reduces interneuronal correlations in macaque visual cortex. Neuron 71, 750–761 (2011).

    Article  CAS  Google Scholar 

  7. Adibi, M., McDonald, J.S., Clifford, C.W. & Arabzadeh, E. Adaptation improves neural coding efficiency despite increasing correlations in variability. J. Neurosci. 33, 2108–2120 (2013).

    Article  CAS  Google Scholar 

  8. Abbott, L.F. & Dayan, P. The effect of correlated variability on the accuracy of a population code. Neural Comput. 11, 91–101 (1999).

    Article  CAS  Google Scholar 

  9. Renart, A. et al. The asynchronous state in cortical circuits. Science 327, 587–590 (2010).

    Article  CAS  Google Scholar 

  10. Cohen, M.R. & Maunsell, J.H. Attention improves performance primarily by reducing interneuronal correlations. Nat. Neurosci. 12, 1594–1600 (2009).

    Article  CAS  Google Scholar 

  11. Mitchell, J.F., Sundberg, K.A. & Reynolds, J.H. Spatial attention decorrelates intrinsic activity fluctuations in macaque area V4. Neuron 63, 879–888 (2009).

    Article  CAS  Google Scholar 

  12. Ecker, A.S. et al. Decorrelated neuronal firing in cortical microcircuits. Science 327, 584–587 (2010).

    Article  CAS  Google Scholar 

  13. Sompolinsky, H., Yoon, H., Kang, K. & Shamir, M. Population coding in neuronal systems with correlated noise. Phys. Rev. E Stat. Nonlin. Soft Matter Phys. 64, 051904 (2001).

    Article  CAS  Google Scholar 

  14. Yoon, H. & Sompolinsky, H. The effect of correlations on the Fisher information of population codes. in Advances in Neural Information Processing Systems (eds. Kearns, M.S., Solla, S. & Cohn, D.A.) 167–173 (MIT Press, Cambridge, Massachusetts, 1999).

  15. Shamir, M. & Sompolinsky, H. Implications of neuronal diversity on population coding. Neural Comput. 18, 1951–1986 (2006).

    Article  Google Scholar 

  16. Ecker, A.S., Berens, P., Tolias, A.S. & Bethge, M. The effect of noise correlations in populations of diversely tuned neurons. J. Neurosci. 31, 14272–14283 (2011).

    Article  CAS  Google Scholar 

  17. Papoulis, A. Probability, Random Variables and Stochastic Process (McGraw-Hill, New York, 1991).

  18. Paradiso, M.A. A theory of the use of visual orientation information which exploits the columnar structure of striate cortex. Biol. Cybern. 58, 35–49 (1988).

    Article  CAS  Google Scholar 

  19. Graf, A.B., Kohn, A., Jazayeri, M. & Movshon, J.A. Decoding the activity of neuronal populations in macaque primary visual cortex. Nat. Neurosci. 14, 239–245 (2011).

    Article  CAS  Google Scholar 

  20. Berens, P. et al. A fast and simple population code for orientation in primate V1. J. Neurosci. 32, 10618–10626 (2012).

    Article  CAS  Google Scholar 

  21. Maimon, G. & Assad, J.A. Beyond Poisson: increased spike-time regularity across primate parietal cortex. Neuron 62, 426–440 (2009).

    Article  CAS  Google Scholar 

  22. Gershon, E.D., Wiener, M.C., Latham, P.E. & Richmond, B.J. Coding strategies in monkey V1 and inferior temporal cortices. J. Neurophysiol. 79, 1135–1144 (1998).

    Article  CAS  Google Scholar 

  23. Qi, X.L. & Constantinidis, C. Variability of prefrontal neuronal discharges before and after training in a working memory task. PLoS ONE 7, e41053 (2012).

    Article  CAS  Google Scholar 

  24. Churchland, M.M. et al. Stimulus onset quenches neural variability: a widespread cortical phenomenon. Nat. Neurosci. 13, 369–378 (2010).

    Article  CAS  Google Scholar 

  25. Beck, J., Bejjanki, V.R. & Pouget, A. Insights from a simple expression for linear fisher information in a recurrently connected population of spiking neurons. Neural Comput. 23, 1484–1502 (2011).

    Article  Google Scholar 

  26. Churchland, M.M. & Shenoy, K.V. Temporal complexity and heterogeneity of single-neuron activity in premotor and motor cortex. J. Neurophysiol. 97, 4235–4257 (2007).

    Article  Google Scholar 

  27. Berens, P., Ecker, A.S., Gerwinn, S., Tolias, A.S. & Bethge, M. Reassessing optimal neural population codes with neurometric functions. Proc. Natl. Acad. Sci. USA 108, 4423–4428 (2011).

    Article  CAS  Google Scholar 

  28. Beck, J.M., Ma, W.J., Pitkow, X., Latham, P.E. & Pouget, A. Not noisy, just wrong: the role of suboptimal inference in behavioral variability. Neuron 74, 30–39 (2012).

    Article  CAS  Google Scholar 

  29. Shadlen, M.N. & Newsome, W.T. The variable discharge of cortical neurons: Implications for connectivity, computation and information coding. J. Neurosci. 18, 3870–3896 (1998).

    Article  CAS  Google Scholar 

  30. Romo, R., Hernandez, A., Zainos, A. & Salinas, E. Correlated neuronal discharges that increase coding efficiency during perceptual discrimination. Neuron 38, 649–657 (2003).

    Article  CAS  Google Scholar 

  31. Goris, R.L., Movshon, J.A. & Simoncelli, E.P. Partitioning neuronal variability. Nat. Neurosci. 17, 858–865 (2014).

    Article  CAS  Google Scholar 

  32. Ecker, A.S. et al. State dependence of noise correlations in macaque primary visual cortex. Neuron 82, 235–248 (2014).

    Article  CAS  Google Scholar 

  33. Kohn, A. & Smith, M.A. Stimulus dependence of neuronal correlation in primary visual cortex of the macaque. J. Neurosci. 25, 3661–3673 (2005).

    Article  CAS  Google Scholar 

  34. Wimmer, K., Nykamp, D.Q., Constantinidis, C. & Compte, A. Bump attractor dynamics in prefrontal cortex explains behavioral precision in spatial working memory. Nat. Neurosci. 17, 431–439 (2014).

    Article  CAS  Google Scholar 

  35. Huang, X. & Lisberger, S.G. Noise correlations in cortical area MT and their potential impact on trial-by-trial variation in the direction and speed of smooth-pursuit eye movements. J. Neurophysiol. 101, 3012–3030 (2009).

    Article  Google Scholar 

  36. Ponce-Alvarez, A., Thiele, A., Albright, T.D., Stoner, G.R. & Deco, G. Stimulus-dependent variability and noise correlations in cortical MT neurons. Proc. Natl. Acad. Sci. USA 110, 13162–13167 (2013).

    Article  CAS  Google Scholar 

  37. Newsome, W.T., Britten, K.H. & Movshon, J.A. Neuronal correlates of a perceptual decision. Nature 341, 52–54 (1989).

    Article  CAS  Google Scholar 

  38. Shadlen, M.N., Britten, K.H., Newsome, W.T. & Movshon, T.A. A computational analysis of the relationship between neuronal and behavioral responses to visual motion. J. Neurosci. 16, 1486–1510 (1996).

    Article  CAS  Google Scholar 

  39. Series, P., Latham, P. & Pouget, A. Tuning curve sharpening for orientation selectivity: coding efficiency and the impact of correlations. Nat. Neurosci. 7, 1129–1135 (2004).

    Article  CAS  Google Scholar 

  40. Barlow, H. Redundancy reduction revisited. Network 12, 241–253 (2001).

    Article  CAS  Google Scholar 

  41. Beck, J., Bejjanki, V.R. & Pouget, A. Insights from a simple expression for linear Fisher information in a recurrently connected population of spiking neurons. Neural Comput. 23, 1484–1502 (2011).

    Article  Google Scholar 

Download references

Acknowledgements

R.M.-B. was supported by the Ramón y Cajal Spanish Award RYC-2010-05952 and by the Marie Curie FP7-PEOPLE-2010-IRG grant PIRG08-GA-2010-276795. X.P. was supported in part by US National Institutes of Health grant T32DC009974. P.L. was supported by the Gatsby Charitable Foundation. A.P. was supported by a grant from the Swiss National Science Foundation (#31003A_143707), and a grant from the Human Frontier Science Program.

Author information

Authors and Affiliations

  1. Research Unit, Parc Sanitari Sant Joan de Déu and Universitat de Barcelona, Esplugues de Llobregat, Barcelona, Spain

    Rubén Moreno-Bote

  2. Centro de Investigación Biomédica en Red de Salud Mental (CIBERSAM), Esplugues de Llobregat, Barcelona, Spain

    Rubén Moreno-Bote

  3. Department of Brain and Cognitive Sciences, University of Rochester, Rochester, New York, USA

    Jeffrey Beck, Xaq Pitkow & Alexandre Pouget

  4. Department of Basic Neuroscience, University of Geneva, Geneva, Switzerland

    Ingmar Kanitscheider & Alexandre Pouget

  5. Department of Neuroscience, Baylor College of Medicine, Houston, Texas, USA

    Xaq Pitkow

  6. Department of Electrical and Computer Engineering, Rice University, Houston, Texas, USA

    Xaq Pitkow

  7. Gatsby Computational Neuroscience Unit, University College London, London, UK

    Peter Latham & Alexandre Pouget

Authors
  1. Rubén Moreno-Bote
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jeffrey Beck
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ingmar Kanitscheider
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xaq Pitkow
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Peter Latham
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Alexandre Pouget
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

R.M.-B., J.B., P.L. and A.P. conceived the project. R.M.-B., J.B., I.K., X.P., P.L. and A.P. developed the theory and wrote the manuscript.

Corresponding author

Correspondence to Alexandre Pouget.

Ethics declarations

Competing interests

The authors declare no competing financial interests.

Integrated supplementary information

Supplementary Figure 1 Effect of observation time.

Output information for a non-leaky integrate and fire network as a function of network size, N for several values of the observation time, T :100 ms (pink), 500 ms (purple), 2 seconds (red), and 10 seconds (green). The black solid line is the information in the input, Eq. (25).Parameters correspond to those of the network with high correlations (Table 1 with JIE = 5.4 and JII = 8.9); this is the same network we used to make Fig. 2d of the main text. The open green circles here correspond to the ones in Fig. 2d of the main text; slight differences between the open green circles in this figure and in Fig. 2d are because we used a different seed in the random number generator in the two sets of simulations; those differences are within error bars. Small observation times reduce information, but the effect diminishes as the number of neurons increases.

Supplementary Figure 2 Visualizing information limiting correlations.

All figures lie in firing rate space, here shown as two (out of N) dimensional. Red indicates the noise; in panels a, c and d, it denotes the noise covariance ellipse. We are assuming that one of the eigenvalues of the noise covariance ellipse is O(N) and the rest are O(1).

a. When the angle between f′(s) and the long (i.e, O(N)) direction of the noise covariance ellipse is O(1/N1/2), the correlations are information limiting.

b. Realistic information limiting noise tracks f(s). The thin black curve is the covariance ellipse from panel a.

c. When there are information limiting correlations, a suboptimal linear decoder, w, predicts that information saturates with N even if the angle between w and the long direction of the noise covariance ellipse is O(1).

d. When there are no information limiting correlations, so that information is O(N), the same linear decoder still predicts that information saturates with N. To observe the O(N) scaling, w would have to be nearly perpendicular to the long direction of the covariance ellipse.

Supplementary Figure 3 Empirically estimated information using ridge regularization.

The black lines on the right show the upper bound on information given by 1/ò (see Eq. (64)); the dashed blue lines are the true information. Different colors indicate different number of samples (M) used to estimate information: from top to bottom, M is 400, 800, 1600, 2400, 3200, and 4000. On the left, differential correlations are not present and information grows with the number of neurons. On the right, differential correlations are present and information saturates at the black line.

First row: The ridge parameter, λ, is fixed at 0.1.

Second row: The ridge parameter is chosen so that the effective number of degrees of freedom of the resulting covariance matrix is 90\% of the maximum number of degrees of freedom possible (see Eq. (107)); typically λ = 0.25.

Third row: The inverse of the variance of a decoder regularized using ridge regression and optimized by minimizing cross-validation error.

Supplementary information

Supplementary Text and Figures

Supplementary Figures 1–3 and Supplementary Modeling (PDF 2065 kb)

Supplementary Methods Checklist (PDF 346 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Moreno-Bote, R., Beck, J., Kanitscheider, I. et al. Information-limiting correlations. Nat Neurosci 17, 1410–1417 (2014). https://doi.org/10.1038/nn.3807

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/nn.3807

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing