[go: up one dir, main page]
Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

Insights into the accuracy of social scientists’ forecasts of societal change

Nature Human Behaviour volume 7, pages 484–501 (2023)Cite this article

Subjects

Abstract

How well can social scientists predict societal change, and what processes underlie their predictions? To answer these questions, we ran two forecasting tournaments testing the accuracy of predictions of societal change in domains commonly studied in the social sciences: ideological preferences, political polarization, life satisfaction, sentiment on social media, and gender–career and racial bias. After we provided them with historical trend data on the relevant domain, social scientists submitted pre-registered monthly forecasts for a year (Tournament 1; N = 86 teams and 359 forecasts), with an opportunity to update forecasts on the basis of new data six months later (Tournament 2; N = 120 teams and 546 forecasts). Benchmarking forecasting accuracy revealed that social scientists’ forecasts were on average no more accurate than those of simple statistical models (historical means, random walks or linear regressions) or the aggregate forecasts of a sample from the general public (N = 802). However, scientists were more accurate if they had scientific expertise in a prediction domain, were interdisciplinary, used simpler models and based predictions on prior data.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on SpringerLink
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Social scientists’ average forecasting errors, compared against different benchmarks.
Fig. 2: Forecasts and ground truth—are forecasts anchoring on the last few historical data points?
Fig. 3: Ratios of forecasting errors among benchmarks compared to scientific forecasts.
Fig. 4: Cross-tournament consistency in the ranking of domains in terms of forecasting inaccuracy.
Fig. 5: Forecasting errors by prediction approach.
Fig. 6: Contributions of specific forecasting strategies and team characteristics to forecasting accuracy.

Similar content being viewed by others

Data availability

All data used in the main text and supplementary analysis are accessible on GitHub (https://github.com/grossmania/Forecasting-Tournament). All prior data presented to the forecasters are available at https://predictions.uwaterloo.ca/. Historical and ground truth markers were obtained from Project FiveThirtyEight (https://projects.fivethirtyeight.com/polls/generic-ballot), Gallup (https://news.gallup.com/poll/203198/presidential-approval-ratings-donald-trump.aspx), Project Implicit (see the Open Science Framework website at https://osf.io/t4bnj) and the US Census Bureau (https://www.census.gov/data/tables/time-series/demo/popest/2010s-national-detail.html).

Code availability

Our project page at https://github.com/grossmania/Forecasting-Tournament displays all code from this paper. See the Reporting Summary for the R packages and their versions.

References

  1. Hutcherson, C. et al. On the accuracy, media representation, and public perception of psychological scientists’ judgments of societal change. Preprint at https://doi.org/10.31234/osf.io/g8f9s (2023).

  2. Collins, H. & Evans, R. Rethinking Expertise (Univ. of Chicago Press, 2009).

  3. Fama, E. F. Efficient capital markets: a review of theory and empirical work. J. Finance 25, 383–417 (1970).

    Article  Google Scholar 

  4. Tetlock, P. E. Expert Political Judgement: How Good Is It? (Princeton University Press, 2017).

  5. Hofman, J. M. et al. Integrating explanation and prediction in computational social science. Nature 595, 181–188 (2021).

    Article  CAS  PubMed  Google Scholar 

  6. Mandel, D. R. & Barnes, A. Accuracy of forecasts in strategic intelligence. Proc. Natl Acad. Sci. USA 111, 10984–10989 (2014).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  7. Makridakis, S., Spiliotis, E. & Assimakopoulos, V. The M4 Competition: 100,000 time series and 61 forecasting methods. Int. J. Forecast. 36, 54–74 (2020).

    Article  Google Scholar 

  8. Open Science Collaboration. Estimating the reproducibility of psychological science. Science 349, aac4716 (2015).

  9. Hofman, J. M., Sharma, A. & Watts, D. J. Prediction and explanation in social systems. Science 355, 486–488 (2017).

    Article  CAS  PubMed  Google Scholar 

  10. Yarkoni, T. & Westfall, J. Choosing prediction over explanation in psychology: lessons from machine learning. Perspect. Psychol. Sci. 12, 1100–1122 (2017).

    Article  PubMed  PubMed Central  Google Scholar 

  11. Fincher, C. L. & Thornhill, R. Parasite-stress promotes in-group assortative sociality: the cases of strong family ties and heightened religiosity. Behav. Brain Sci. 35, 61–79 (2012).

    Article  PubMed  Google Scholar 

  12. Varnum, M. E. W. & Grossmann, I. Pathogen prevalence is associated with cultural changes in gender equality. Nat. Hum. Behav. 1, 0003 (2016).

    Article  Google Scholar 

  13. Schaller, M. & Murray, D. R. Pathogens, personality, and culture: disease prevalence predicts worldwide variability in sociosexuality, extraversion, and openness to experience. J. Pers. Soc. Psychol. 95, 212–221 (2008).

    Article  PubMed  Google Scholar 

  14. van Leeuwen, F., Park, J. H., Koenig, B. L. & Graham, J. Regional variation in pathogen prevalence predicts endorsement of group-focused moral concerns. Evol. Hum. Behav. 33, 429–437 (2012).

    Article  Google Scholar 

  15. Hawkley, L. C. & Cacioppo, J. T. Loneliness matters: a theoretical and empirical review of consequences and mechanisms. Ann. Behav. Med. 40, 218–227 (2010).

    Article  PubMed  Google Scholar 

  16. Salganik, M. J. et al. Measuring the predictability of life outcomes with a scientific mass collaboration. Proc. Natl Acad. Sci. USA 117, 8398–8403 (2020).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  17. Liberman, M. Reproducible Research and the Common Task Method (2015); https://www.simonsfoundation.org/event/reproducible-research-and-the-common-task-method/

  18. Hyndman, R. J. & Koehler, A. B. Another look at measures of forecast accuracy. Int. J. Forecast. 22, 679–688 (2006).

    Article  Google Scholar 

  19. Eyal, P., David, R., Andrew, G., Zak, E. & Ekaterina, D. Data quality of platforms and panels for online behavioral research. Behav. Res. Methods https://doi.org/10.3758/s13428-021-01694-3 (2021).

  20. Genz, A. & Bretz, F. Computation of Multivariate Normal and t Probabilities (Springer, 2009).

  21. Green, K. C. & Armstrong, J. S. Simple versus complex forecasting: the evidence. J. Bus. Res. 68, 1678–1685 (2015).

    Article  Google Scholar 

  22. Grossmann, I., Twardus, O., Varnum, M. E. W., Jayawickreme, E. & McLevey, J. Expert predictions of societal change: insights from the World After COVID Project. Am. Psychol. 77, 276–290 (2022).

    Article  PubMed  Google Scholar 

  23. Grossmann, I., Huynh, A. C. & Ellsworth, P. C. Emotional complexity: clarifying definitions and cultural correlates. J. Pers. Soc. Psychol. 111, 895–916 (2016).

    Article  PubMed  Google Scholar 

  24. Alves, H., Koch, A. & Unkelbach, C. Why good is more alike than bad: processing implications. Trends Cogn. Sci. 21, 69–79 (2017).

    Article  PubMed  Google Scholar 

  25. Dimant, E. et al. Politicizing mask-wearing: predicting the success of behavioral interventions among Republicans and Democrats in the U.S. Sci. Rep. 12, 7575 (2022).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  26. Dunning, D., Heath, C. & Suls, J. M. Flawed self-assessment. Psychol. Sci. Public Interest 5, 69–106 (2004).

    Article  PubMed  Google Scholar 

  27. Grossmann, I. et al. The science of wisdom in a polarized world: knowns and unknowns. Psychol. Inq. 31, 103–133 (2020).

    Article  Google Scholar 

  28. Porter, T. et al. Predictors and consequences of intellectual humility. Nat. Rev. Psychol. 1, 524–536 (2022).

    Article  PubMed  PubMed Central  Google Scholar 

  29. Mellers, B., Tetlock, P. E. & Arkes, H. R. Forecasting tournaments, epistemic humility and attitude depolarization. Cognition 188, 19–26 (2019).

    Article  PubMed  Google Scholar 

  30. Grossmann, I. et al. Training for wisdom: the distanced-self-reflection diary method. Psychol. Sci. 32, 381–394 (2021).

    Article  PubMed  Google Scholar 

  31. Klein, R. A. et al. Many Labs 2: investigating variation in replicability across samples and settings. Adv. Methods Pract. Psychol. Sci. 1, 443–490 (2018).

    Article  Google Scholar 

  32. Voslinsky, A. & Azar, O. H. Incentives in experimental economics. J. Behav. Exp. Econ. 93, 101706 (2021).

    Article  Google Scholar 

  33. Cerasoli, C. P., Nicklin, J. M. & Ford, M. T. Intrinsic motivation and extrinsic incentives jointly predict performance: a 40-year meta-analysis. Psychol. Bull. 140, 980–1008 (2014).

    Article  PubMed  Google Scholar 

  34. Richard, F. D., Bond, C. F. Jr. & Stokes-Zoota, J. J. One hundred years of social psychology quantitatively described. Rev. Gen. Psychol. 7, 331–363 (2003).

    Article  Google Scholar 

  35. Henrich, J., Heine, S. J. & Norenzayan, A. The weirdest people in the world? Behav. Brain Sci. 33, 61–83 (2010).

    Article  PubMed  Google Scholar 

  36. Yarkoni, T. The generalizability crisis. Behav. Brain Sci. 45, e1 (2022).

    Article  Google Scholar 

  37. Cesario, J. What can experimental studies of bias tell us about real-world group disparities? Behav. Brain Sci. https://doi.org/10.1017/S0140525X21000017 (2021).

  38. IJzerman, H. et al. Use caution when applying behavioural science to policy. Nat. Hum. Behav. 4, 1092–1094 (2020).

    Article  PubMed  Google Scholar 

  39. Varnum, M. E. W. & Grossmann, I. Cultural change: the how and the why. Perspect. Psychol. Sci. 12, 956–972 (2017).

    Article  PubMed  Google Scholar 

  40. Breiman, L. Statistical modeling: the two cultures (with comments and a rejoinder by the author). Stat. Sci. 16, 199–231 (2001).

  41. Lewin, K. Defining the ‘field at a given time’. Psychol. Rev. 50, 292–310 (1943).

    Article  Google Scholar 

  42. Turchin, P., Currie, T. E., Turner, E. A. L. & Gavrilets, S. War, space, and the evolution of Old World complex societies. Proc. Natl Acad. Sci. USA 110, 16384–16389 (2013).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  43. Brockwell, P. J. & Davis, R. A. Introduction to Time Series and Forecasting (Springer, 2016); https://doi.org/10.1007/978-3-319-29854-2

  44. Makridakis, S. & Taleb, N. Living in a world of low levels of predictability. Int. J. Forecast. 25, 840–844 (2009).

    Article  Google Scholar 

  45. Hitchens, N. M., Brooks, H. E. & Kay, M. P. Objective limits on forecasting skill of rare events. Weather Forecast. 28, 525–534 (2013).

    Article  Google Scholar 

  46. Jebb, A. T., Tay, L., Wang, W. & Huang, Q. Time series analysis for psychological research: examining and forecasting change. Front. Psychol. 6, 727 (2015).

    Article  PubMed  PubMed Central  Google Scholar 

  47. Van Bavel, J. et al. Using social and behavioural science to support COVID-19 pandemic response. Nat. Hum. Behav. 4, 460–471 (2020).

    Article  PubMed  Google Scholar 

  48. Seitz, B. M. et al. The pandemic exposes human nature: 10 evolutionary insights. Proc. Natl Acad. Sci. USA 117, 27767–27776 (2020).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  49. Schaller, M. & Park, J. H. The behavioral immune system (and why it matters). Curr. Dir. Psychol. Sci. 20, 99–103 (2011).

    Article  Google Scholar 

  50. Wang, I. M., Michalak, N. M. & Ackerman, J. M. in The SAGE Handbook of Personality and Individual Differences: Origins of Personality and Individual Differences Vol. 2 (eds Zeigler-Hill, V. & Shackelford, T. K.) 321–345 (2018); https://doi.org/10.4135/9781526451200.n18

  51. Luhmann, M. Using Big Data to study subjective well-being. Curr. Opin. Behav. Sci. 18, 28–33 (2017).

    Article  Google Scholar 

  52. Schwartz, H. A. et al. Predicting individual well-being through the language of social media. Biocomputing 2016 https://doi.org/10.1142/9789814749411_0047 (2016).

  53. Kiritchenko, S., Zhu, X. & Mohammad, S. M. Sentiment analysis of short informal texts. J. Artif. Intell. Res. 50, 723–762 (2014).

    Article  Google Scholar 

  54. Witters, D. & Harter, J. In U.S., Life Ratings Plummet to 12-Year Low (2020); https://news.gallup.com/poll/391331/life-ratings-drop-month-low.aspx

  55. Axt, J. R. The best way to measure explicit racial attitudes is to ask about them. Soc. Psychol. Pers. Sci. 9, 896–906 (2018).

    Article  Google Scholar 

  56. Nosek, B. A. et al. Pervasiveness and correlates of implicit attitudes and stereotypes. Eur. Rev. Soc. Psychol. 18, 36–88 (2007).

    Article  Google Scholar 

  57. Hehman, E., Flake, J. K. & Calanchini, J. Disproportionate use of lethal force in policing is associated with regional racial biases of residents. Soc. Psychol. Pers. Sci. 9, 393–401 (2018).

    Article  Google Scholar 

  58. Ofosu, E. K., Chambers, M. K., Chen, J. M. & Hehman, E. Same-sex marriage legalization associated with reduced implicit and explicit antigay bias. Proc. Natl Acad. Sci. USA 116, 8846–8851 (2019).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  59. Charlesworth, T. E. S. & Banaji, M. R. Patterns of implicit and explicit attitudes: I. Long-term change and stability from 2007 to 2016. Psychol. Sci. 30, 174–192 (2019).

    Article  PubMed  Google Scholar 

  60. Greenwald, A. G., Nosek, B. A. & Banaji, M. R. Understanding and using the Implicit Association Test: I. An improved scoring algorithm. J. Pers. Soc. Psychol. 85, 197–216 (2003).

    Article  PubMed  Google Scholar 

  61. Gobet, F. The future of expertise: the need for a multidisciplinary approach. J. Expertise 1, 107–113 (2018).

    Google Scholar 

  62. Lenth, R., Singmann, H., Love, J. & Maxime, H. emmeans: Estimated marginal means, aka least-squares means. R package version 1.8.0 (2020).

  63. R Core Team. R: A Language and Environment for Statistical Computing (2022).

  64. Gelman, A. Scaling regression inputs by dividing by two standard deviations. Stat. Med. 27, 2865–2873 (2008).

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

This programme of research was supported by the Basic Research Program at the National Research University Higher School of Economics (M. Fabrykant), John Templeton Foundation grant no. 62260 (I.G. and P.E.T.), Kega 079UK-4/2021 (P.K.), Ministerio de Ciencia e Innovación España grants no. PID2019-111512RB-I00-HMDM and no. HDL-HS-280218 (A.A.), the National Center for Complementary & Integrative Health of the National Institutes of Health under award no. K23AT010879 (S.B.G.), National Science Foundation RAPID grant no. 2026854 (M.E.W.V.), PID2019-111512RB-I00 (M.S.), NPO Systemic Risk Institute grant no. LX22NPO5101 (I.R.), the Slovak Research and Development Agency under contract no. APVV-20-0319 (M.A.), Social Sciences and Humanities Research Council of Canada Insight grant no. 435-2014-0685 (I.G.), Social Sciences and Humanities Research Council of Canada Connection grant no. 611-2020-0190 (I.G.), and Swiss National Science Foundation grant no. PP00P1_170463 (O. Strijbis). The funders had no role in study design, data collection and analysis, decision to publish or preparation of the manuscript. We thank J. Axt for providing monthly estimates of Project Implicit data and the members of the Forecasting Collaborative who chose to remain anonymous for their contribution to the tournaments.

Author information

Authors and Affiliations

  1. Department of Psychology, University of Waterloo, Waterloo, Ontario, Canada

    Igor Grossmann, Amanda Rotella, Konstantyn Sharpinskyi, Dillon T. Browne & Geoffrey T. Fong

  2. Department of Psychology, Northumbria University, Northumbria, UK

    Amanda Rotella

  3. Department of Psychology, University of Toronto Scarborough, Toronto, Ontario, Canada

    Cendri A. Hutcherson

  4. Department of Psychology, Arizona State University, Tempe, AZ, USA

    Michael E. W. Varnum

  5. Institute of Management Accounting and Simulation, Hamburg University of Technology, Hamburg, Germany

    Sebastian Achter

  6. Department of Psychology, Middlesex University London, London, UK

    Mandeep K. Dhami

  7. Department of Experimental Psychology, University of California, San Diego, San Diego, CA, USA

    Xinqi Evie Guo

  8. Department of Psychology, Toronto Metropolitan University, Toronto, Ontario, Canada

    Mane Kara-Yakoubian

  9. Defence Research and Development Canada, Toronto, Ontario, Canada

    David R. Mandel & Robert N. Collins

  10. Department of Psychology, York University, Toronto, Ontario, Canada

    David R. Mandel

  11. Department of Economics, Tilburg University, Tilburg, the Netherlands

    Louis Raes

  12. Department of Psychological Sciences, Purdue University, West Lafayette, IN, USA

    Louis Tay

  13. Mathematical Institute, University of Oxford, Oxford, UK

    Aymeric Vie

  14. Institute of New Economic Thinking, University of Oxford, Oxford, UK

    Aymeric Vie

  15. Jacobs Center for Productive Youth Development, University of Zurich, Zurich, Switzerland

    Lisa Wagner

  16. Institute of Psychology, University of Prešov, Prešov, Slovakia

    Matus Adamkovic & Gabriel Baník

  17. Institute of Social Sciences, CSPS, Slovak Academy of Sciences, Bratislava, Slovakia

    Matus Adamkovic

  18. Department of Mechanical and Mechatronics Engineering, University of Waterloo, Waterloo, Ontario, Canada

    Arash Arami

  19. Toronto Rehabilitation Institute (KITE), University Health Network, Toronto, Canada

    Arash Arami

  20. Iscte-University Institute of Lisbon, CIS, Lisbon, Portugal

    Patrícia Arriaga

  21. Melbourne Centre for Data Science, University of Melbourne, Melbourne, Victoria, Australia

    Kasun Bandara

  22. Faculty of Social and Behavioural Sciences, University of Amsterdam, Amsterdam, the Netherlands

    František Bartoš & Shiva Pauer

  23. Department of Food Marketing, Haub School of Business, Saint Joseph’s University, Philadelphia, PA, USA

    Ernest Baskin

  24. Department of Data Science and Artificial Intelligence, Monash University, Melbourne, Victoria, Australia

    Christoph Bergmeir

  25. Institute of Psychology, University of Wrocław, Wrocław, Poland

    Michał Białek & Michal Misiak

  26. Department of Management, Aarhus University, Aarhus, Denmark

    Caroline K. Børsting & Christian T. Elbaek

  27. Anderson School of Management, University of California, Los Angeles, Los Angeles, CA, USA

    Eugene M. Caruso

  28. Department of Psychology, Dominican University of California, San Rafael, CA, USA

    Rong Chen

  29. Department of Industrial Economics, Tamkang University, New Taipei City, Taiwan

    Bin-Tzong Chie

  30. Department of Psychology, Michigan State University, East Lansing, MI, USA

    William J. Chopik

  31. Independent Researcher, Penang, Malaysia

    Chin Wen Cong

  32. Psychology Department, Grove City College, Grove City, PA, USA

    Lucian G. Conway

  33. Department of Economics, Siena College, Loudonville, NY, USA

    Matthew Davis

  34. Department of Psychology, Memorial University of Newfoundland, St. John’s, Newfoundland, Canada

    Martin V. Day

  35. UBC Sauder School of Business, University of British Columbia, Vancouver, British Columbia, Canada

    Nathan A. Dhaliwal

  36. Department of Psychology, University of Oklahoma, Norman, OK, USA

    Justin D. Durham & Scott D. Gronlund

  37. Faculty of Psychology and Cognitive Science, Adam Mickiewicz University, Poznań, Poland

    Martyna Dziekan

  38. Department of Psychology, University of Groningen, Groningen, the Netherlands

    Eric Shuman

  39. Laboratory for Comparative Studies in Mass Consciousness, Expert Institute, HSE University, Moscow, Russia

    Marharyta Fabrykant

  40. Faculty of Philosophy and Social Sciences, Belarusian State University, Minsk, Belarus

    Marharyta Fabrykant

  41. Department of Sociology, Radboud University, Nijmegen, the Netherlands

    Mustafa Firat

  42. Ontario Institute for Cancer Research, Toronto, Ontario, Canada

    Geoffrey T. Fong

  43. Department of Psychology, University of Winnipeg, Winnipeg, Manitoba, Canada

    Jeremy A. Frimer

  44. Department of Psychology, University of Washington, Seattle, WA, USA

    Jonathan M. Gallegos, Cheryl R. Kaiser & Adam H. Smiley

  45. Department of Counseling Psychology, University of Wisconsin–Madison, Madison, WI, USA

    Simon B. Goldberg

  46. Department of Leadership and Organizational Behaviour, BI Norwegian Business School, Oslo, Norway

    Anton Gollwitzer

  47. Center for Adaptive Rationality, Max Planck Institute for Human Development, Berlin, Germany

    Anton Gollwitzer

  48. School of Public Health Sciences, University of Waterloo, Waterloo, Ontario, Canada

    Julia Goyal

  49. TU Dortmund University, Dortmund, Germany

    Lorenz Graf-Vlachy

  50. ESCP Business School, Paris, France

    Lorenz Graf-Vlachy

  51. IESE Business School, Barcelona, Spain

    Sebastian Hafenbrädl

  52. School of Social Sciences, Singapore Management University, Singapore, Singapore

    Andree Hartanto & Verity Y. Q. Lua

  53. Center for Healthy Minds, University of Wisconsin–Madison, Madison, WI, USA

    Matthew J. Hirshberg

  54. University of Queensland Business School, Brisbane, Queensland, Australia

    Matthew J. Hornsey

  55. Melbourne School of Psychological Sciences, University of Melbourne, Melbourne, Victoria, Australia

    Piers D. L. Howe & Marcellin F. R. C. Martinie

  56. Department of Marketing, University of Massachusetts Dartmouth, Dartmouth, MA, USA

    Anoosha Izadi

  57. Department of Experimental and Applied Psychology, Vrije Universiteit Amsterdam, Amsterdam, the Netherlands

    Bastian Jaeger, Joshua M. Tybur & Alberto López Ortega

  58. Department of Psychology, Faculty of Arts, Pavol Jozef Šafárik University in Košice, Košice, Slovakia

    Pavol Kačmár

  59. Cambridge Judge Business School, University of Cambridge, Cambridge, UK

    Yeun Joon Kim

  60. Hermes Germany GmbH, Hamburg, Germany

    Ruslan Krenzler

  61. University of Hamburg, Hamburg, Germany

    Ruslan Krenzler

  62. Department of Psychology, Illinois State University, Normal, IL, USA

    Daniel G. Lannin

  63. Department of Business Administration, National Pingtung University, Pingtung City, Taiwan

    Hung-Wen Lin

  64. Department of Psychology, University of Victoria, Victoria, British Columbia, Canada

    Nigel Mantou Lou

  65. Centre for Youth and Society, University of Victoria, Victoria, British Columbia, Canada

    Nigel Mantou Lou

  66. Department of Psychology, California State University, Fullerton, Fullerton, CA, USA

    Aaron W. Lukaszewski

  67. Center for the Study of Human Nature, California State University, Fullerton, Fullerton, CA, USA

    Aaron W. Lukaszewski

  68. Department of Psychology, Loma Linda University, Loma Linda, CA, USA

    Albert L. Ly & Jaime M. Napan

  69. University of Nottingham, Nottingham, UK

    Christopher R. Madan

  70. Department of Experimental Psychology, University College London, London, UK

    Maximilian Maier

  71. Singapore Management University, Singapore, Singapore

    Nadyanna M. Majeed

  72. Department of Psychology, Florida State University, Tallahassee, FL, USA

    David S. March

  73. Department of Psychology, Georgetown University, Washington, DC, USA

    Abigail A. Marsh

  74. School of Anthropology & Museum Ethnography, University of Oxford, Oxford, UK

    Michal Misiak

  75. School for Business and Society, University of York, York, UK

    Kristian Ove R. Myrseth

  76. Department of Psychology, Columbia University, New York, NY, USA

    Jonathan Nicholas, Paul A. Bloom & Margaux N. A. Wienk

  77. IHRR Forecasting Laboratory, Durham University, Durham, UK

    Konstantinos Nikolopoulos

  78. Department of Psychology, Aberystwyth University, Aberystwyth, UK

    Jiaqing O

  79. School of Business and Law, Department of Management, University of Agder, Kristiansand, Norway

    Tobias Otterbring

  80. Institute of Retail Economics, Stockholm, Sweden

    Tobias Otterbring

  81. Institute of Psychology, University of Silesia in Katowice, Katowice, Poland

    Mariola Paruzel-Czachura

  82. Department of Neurology, Penn Center for Neuroaesthetics, University of Pennsylvania, Philadelphia, PA, USA

    Mariola Paruzel-Czachura

  83. Central Connecticut State University, New Britain, CT, USA

    John Protzko

  84. Department of Psychology, University of Arizona, Tucson, AZ, USA

    Quentin Raffaelli & Jessica R. Andrews-Hanna

  85. Faculty of Education, Institute for Research and Development of Education, Charles University, Prague, Czech Republic

    Ivan Ropovik

  86. Faculty of Education, University of Prešov, Prešov, Slovakia

    Ivan Ropovik

  87. School of Psychology, Macquarie University, Sydney, New South Wales, Australia

    Robert M. Ross

  88. Department of Human Service, University of Haifa, Haifa, Israel

    Yefim Roth

  89. Promenta Center, Department of Psychology, University of Oslo, Oslo, Norway

    Espen Røysamb

  90. Department of Sociology, Cornell University, Ithaca, NY, USA

    Landon Schnabel

  91. Institute of Psychology, University of Bamberg, Bamberg, Germany

    Astrid Schütz

  92. IE Business School, IE University, Madrid, Spain

    Matthias Seifert

  93. Faculty of Psychology and Human Movement Science, University of Hamburg, Hamburg, Germany

    A. T. Sevincer

  94. Department of Computer and Information Science, University of Pennsylvania, Philadelphia, PA, USA

    Garrick T. Sherman & Lyle Ungar

  95. Department of Clinical Neuroscience, Karolinska Institutet, Solna, Sweden

    Otto Simonsson

  96. Department of Sociology, University of Oxford, Oxford, UK

    Otto Simonsson

  97. Department of Decision Analytics and Risk, University of Southampton, Southampton, UK

    Ming-Chien Sung & Chung-Ching Tai

  98. University of Chicago Booth School of Business, Chicago, IL, USA

    Thomas Talhelm

  99. Department of Psychology, University of Virginia, Charlottesville, VA, USA

    Bethany A. Teachman

  100. Psychology Department, University of Pennsylvania, Philadelphia, PA, USA

    Philip E. Tetlock

  101. Wharton School of Business, University of Pennsylvania, Philadelphia, PA, USA

    Philip E. Tetlock

  102. Department of Economics, National and Kapodistrian University of Athens, Athens, Greece

    Dimitrios Thomakos

  103. School of Psychological Sciences and Health, University of Strathclyde, Glasgow, UK

    Dwight C. K. Tse

  104. Department of Psychology, University of Guelph, Guelph, Ontario, Canada

    Oliver J. Twardus

  105. Psychology Department, Hebrew University of Jerusalem, Jerusalem, Israel

    Daan Vandermeulen

  106. Department of Economics, Nottingham Trent University, Nottingham, UK

    Leighton Vaughan Williams

  107. Department of Management and Organizations, Northwestern University, Evanston, IL, USA

    Hrag A. Vosgerichian

  108. College of Human Ecology, Cornell University, Ithaca, NY, USA

    Qi Wang

  109. Harvard Kennedy School, Harvard University, Cambridge, MA, USA

    Ke Wang

  110. Computer and Information Science, University of Pennsylvania, Philadelphia, PA, USA

    Mark E. Whiting

  111. Operations, Information, and Decisions Department, the Wharton School, University of Pennsylvania, Philadelphia, PA, USA

    Mark E. Whiting

  112. School of Economics and Finance, University of St. Andrews, St. Andrews, UK

    Conny E. Wollbrant

  113. Department of Management, Cameron School of Business, University of North Carolina Wilmington, Wilmington, NC, USA

    Tao Yang

  114. School of Psychology, Speech and Hearing, University of Canterbury, Christchurch, New Zealand

    Kumar Yogeeswaran

  115. Department of Marketing, University of Dayton, Dayton, OH, USA

    Sangsuk Yoon

  116. ISG Universidade Lusofona, Lisbon, Portugal

    Ventura R. Alves

  117. Cognitive Science, University of Arizona, Tucson, AZ, USA

    Jessica R. Andrews-Hanna

  118. Ephemer AI, Atlanta, GA, USA

    Anthony Boyles

  119. Questrom School of Business, Boston University, Boston, MA, USA

    Loo Charis

  120. Institute of Social Science Research, Pusan National University, Busan, South Korea

    Mingyeong Choi

  121. Fielding School of Public Health, University of California, Los Angeles, Los Angeles, CA, USA

    Sean Darling-Hammond

  122. Psychology Department, University of Washington, Seattle, WA, USA

    Z. E. Ferguson

  123. Department of Political Science, Aarhus University, Aarhus, Denmark

    Simon T. Karg

  124. College of Science and Mathematics, Wright State University, Fairborn, OH, USA

    Lori Mahoney

  125. Department of Psychology, Georgia Southern University, Statesboro, GA, USA

    Melvin S. Marsh

  126. Division of Epidemiology, School of Public Health, University of California, Berkeley, Berkeley, CA, USA

    Eli K. Michaels

  127. Department of Psychology, Uppsala University, Uppsala, Sweden

    Philip Millroth

  128. Department of Psychology, Carnegie Mellon University, Pittsburgh, PA, USA

    Jeanean B. Naqvi

  129. School of Humanities & Behavioral Sciences, Singapore University of Social Sciences, Singapore, Singapore

    Weiting Ng

  130. Department of Psychology, Yale University, New Haven, CT, USA

    Robb B. Rutledge

  131. BehaviourWorks Australia, Monash University, Melbourne, Victoria, Australia

    Peter Slattery

  132. Institute of Political Science, University of Zurich, Zurich, Switzerland

    Oliver Strijbis

  133. Department of Psychology, Oklahoma State University, Stillwater, OK, USA

    Daniel Sznycer

  134. Department of Business Administration, University of Hawaii–West Oahu, Kapolei, HI, USA

    Eli Tsukayama

  135. Department of Sociology, Stanford University, Stanford, CA, USA

    Austin van Loon & Jan G. Voelkel

  136. Department of Economics, University of Melbourne, Melbourne, Victoria, Australia

    Tom Wilkening

Consortia

The Forecasting Collaborative

  • Igor Grossmann
  • , Amanda Rotella
  • , Cendri A. Hutcherson
  • , Konstantyn Sharpinskyi
  • , Michael E. W. Varnum
  • , Sebastian Achter
  • , Mandeep K. Dhami
  • , Xinqi Evie Guo
  • , Mane Kara-Yakoubian
  • , David R. Mandel
  • , Louis Raes
  • , Louis Tay
  • , Aymeric Vie
  • , Lisa Wagner
  • , Matus Adamkovic
  • , Arash Arami
  • , Patrícia Arriaga
  • , Kasun Bandara
  • , Gabriel Baník
  • , František Bartoš
  • , Ernest Baskin
  • , Christoph Bergmeir
  • , Michał Białek
  • , Caroline K. Børsting
  • , Dillon T. Browne
  • , Eugene M. Caruso
  • , Rong Chen
  • , Bin-Tzong Chie
  • , William J. Chopik
  • , Robert N. Collins
  • , Chin Wen Cong
  • , Lucian G. Conway
  • , Matthew Davis
  • , Martin V. Day
  • , Nathan A. Dhaliwal
  • , Justin D. Durham
  • , Martyna Dziekan
  • , Christian T. Elbaek
  • , Eric Shuman
  • , Marharyta Fabrykant
  • , Mustafa Firat
  • , Geoffrey T. Fong
  • , Jeremy A. Frimer
  • , Jonathan M. Gallegos
  • , Simon B. Goldberg
  • , Anton Gollwitzer
  • , Julia Goyal
  • , Lorenz Graf-Vlachy
  • , Scott D. Gronlund
  • , Sebastian Hafenbrädl
  • , Andree Hartanto
  • , Matthew J. Hirshberg
  • , Matthew J. Hornsey
  • , Piers D. L. Howe
  • , Anoosha Izadi
  • , Bastian Jaeger
  • , Pavol Kačmár
  • , Yeun Joon Kim
  • , Ruslan Krenzler
  • , Daniel G. Lannin
  • , Hung-Wen Lin
  • , Nigel Mantou Lou
  • , Verity Y. Q. Lua
  • , Aaron W. Lukaszewski
  • , Albert L. Ly
  • , Christopher R. Madan
  • , Maximilian Maier
  • , Nadyanna M. Majeed
  • , David S. March
  • , Abigail A. Marsh
  • , Michal Misiak
  • , Kristian Ove R. Myrseth
  • , Jaime M. Napan
  • , Jonathan Nicholas
  • , Konstantinos Nikolopoulos
  • , Jiaqing O
  • , Tobias Otterbring
  • , Mariola Paruzel-Czachura
  • , Shiva Pauer
  • , John Protzko
  • , Quentin Raffaelli
  • , Ivan Ropovik
  • , Robert M. Ross
  • , Yefim Roth
  • , Espen Røysamb
  • , Landon Schnabel
  • , Astrid Schütz
  • , Matthias Seifert
  • , A. T. Sevincer
  • , Garrick T. Sherman
  • , Otto Simonsson
  • , Ming-Chien Sung
  • , Chung-Ching Tai
  • , Thomas Talhelm
  • , Bethany A. Teachman
  • , Philip E. Tetlock
  • , Dimitrios Thomakos
  • , Dwight C. K. Tse
  • , Oliver J. Twardus
  • , Joshua M. Tybur
  • , Lyle Ungar
  • , Daan Vandermeulen
  • , Leighton Vaughan Williams
  • , Hrag A. Vosgerichian
  • , Qi Wang
  • , Ke Wang
  • , Mark E. Whiting
  • , Conny E. Wollbrant
  • , Tao Yang
  • , Kumar Yogeeswaran
  • , Sangsuk Yoon
  • , Ventura R. Alves
  • , Jessica R. Andrews-Hanna
  • , Paul A. Bloom
  • , Anthony Boyles
  • , Loo Charis
  • , Mingyeong Choi
  • , Sean Darling-Hammond
  • , Z. E. Ferguson
  • , Cheryl R. Kaiser
  • , Simon T. Karg
  • , Alberto López Ortega
  • , Lori Mahoney
  • , Melvin S. Marsh
  • , Marcellin F. R. C. Martinie
  • , Eli K. Michaels
  • , Philip Millroth
  • , Jeanean B. Naqvi
  • , Weiting Ng
  • , Robb B. Rutledge
  • , Peter Slattery
  • , Adam H. Smiley
  • , Oliver Strijbis
  • , Daniel Sznycer
  • , Eli Tsukayama
  • , Austin van Loon
  • , Jan G. Voelkel
  • , Margaux N. A. Wienk
  •  & Tom Wilkening

Contributions

Conceptualization: I.G., A.R., C.A.H., M.E.W.V., L.T. and P.E.T. Data curation: I.G., K.S., G.T.S. and O.J.T. Forecasting: S.A., M.K.D., X.E.G., M. J. Hirshberg, M.K.-Y., D.R.M., L.R., A.V., L.W., M.A., A.A., P.A., K.B., G.B., F.B., E.B., C.B., M.B., C.K.B., D.T.B., E.M.C., R.C., B.-T.C., W.J.C., C.W.C., L.G.C., M. Davis, M.V.D., N.A.D., J.D.D., M. Dziekan, C.T.E., E.S., M. Fabrykant, M. Firat, G.T.F., J.A.F., J.M.G., S.B.G., A.G., J.G., L.G.-V., S.D.G., S.H., A.H., M. J. Hornsey, P.D.L.H., A.I., B.J., P.K., Y.J.K., R.K., D.G.L., H.-W.L., N.M.L., V.Y.Q.L., A.W.L., A.L.L., C.R.M., M. Maier, N.M.M., D.S.M., A.A.M., M. Misiak, K.O.R.M., J.M.N., J.N., K.N., J.O., T.O., M.P.-C., S.P., J.P., Q.R., I.R., R.M.R., Y.R., E.R., L.S., A.S., M.S., A.T.S., O. Simonsson, M.-C.S., C.-C.T., T.T., B.A.T., D.T., D.C.K.T., J.M.T., L.U., D.V., L.V.W., H.A.V., Q.W., K.W., M.E.W., C.E.W., T.Y., K.Y., S.Y., V.R.A., J.R.A.-H., P.A.B., A.B., L.C., M.C., S.D.-H., Z.E.F., C.R.K., S.T.K., A.L.O., L.M., M.S.M., M.F.R.C.M., E.K.M., P.M., J.B.N., W.N., R.B.R., P.S., A.H.S., O. Strijbis, D.S., E.T., A.v.L., J.G.V., M.N.A.W. and T.W. Formal analysis: I.G. and C.A.H. Funding acquisition: I.G. Investigation: I.G., A.R. and C.A.H. Methodology: I.G., A.R., C.A.H., K.S., M.E.W.V., S.A., D.R.M., L.R., L.T., A.V., R.N.C., L.U. and D.V. Project administration: I.G., A.R., M.E.W.V., M.K.-Y. and O.J.T. Resources: I.G., A.R., J.N. and G.T.S. Supervision: I.G. Validation: K.S., X.E.G. and L.W. Visualization: I.G. and M.K.D. Writing—original draft: I.G. Writing—review and editing: I.G., A.R., C.A.H., K.S., M.E.W.V., S.A., M.K.D., X.E.G., M. J. Hirshberg, M.K.-Y., D.R.M., L.R., L.T., A.V., L.W., M.A., A.A., P.A., K.B., G.B., F.B., E.B., C.B., M.B., C.K.B., D.T.B., E.M.C., R.C., B.-T.C., W.J.C., R.N.C., C.W.C., L.G.C., M. Davis, M.V.D., N.A.D., J.D.D., M. Dziekan, C.T.E., E.S., M. Fabrykant, M. Firat, G.T.F., J.A.F., J.M.G., S.B.G., A.G., J.G., L.G.-V., S.D.G., S.H., A.H., M. J. Hornsey, P.D.L.H., A.I., B.J., P.K., Y.J.K., R.K., D.G.L., H.-W.L., N.M.L., V.Y.Q.L., A.W.L., A.L.L., C.R.M., M. Maier, N.M.M., D.S.M., A.A.M., M. Misiak, K.O.R.M., J.M.N., K.N., J.O., T.O., M.P.-C., S.P., J.P., Q.R., I.R., R.M.R., Y.R., E.R., L.S., A.S., M.S., A.T.S., O. Simonsson, M.-C.S., C.-C.T., T.T., B.A.T., P.E.T., D.T., D.C.K.T., J.M.T., L.V.W., H.A.V., Q.W., K.W., M.E.W., C.E.W., T.Y., K.Y. and S.Y.

Ethics declarations

Competing interests

The authors declare no competing interests.

Peer review

Peer review information

Nature Human Behaviour thanks Richard Klein and the other, anonymous, reviewer(s) for their contribution to the peer review of this work. Peer reviewer reports are available.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary Information

Supplementary Methods, Figs. 1–15, Tables 1–9 and Appendices 1–5.

Reporting Summary

Peer Review File

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

The Forecasting Collaborative. Insights into the accuracy of social scientists’ forecasts of societal change. Nat Hum Behav 7, 484–501 (2023). https://doi.org/10.1038/s41562-022-01517-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41562-022-01517-1

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing