[go: up one dir, main page]
IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2411.18830.html
   My bibliography  Save this paper

Double Descent in Portfolio Optimization: Dance between Theoretical Sharpe Ratio and Estimation Accuracy

Author

Listed:
  • Yonghe Lu
  • Yanrong Yang
  • Terry Zhang
Abstract
We study the relationship between model complexity and out-of-sample performance in the context of mean-variance portfolio optimization. Representing model complexity by the number of assets, we find that the performance of low-dimensional models initially improves with complexity but then declines due to overfitting. As model complexity becomes sufficiently high, the performance improves with complexity again, resulting in a double ascent Sharpe ratio curve similar to the double descent phenomenon observed in artificial intelligence. The underlying mechanisms involve an intricate interaction between the theoretical Sharpe ratio and estimation accuracy. In high-dimensional models, the theoretical Sharpe ratio approaches its upper limit, and the overfitting problem is reduced because there are more parameters than data restrictions, which allows us to choose well-behaved parameters based on inductive bias.

Suggested Citation

  • Yonghe Lu & Yanrong Yang & Terry Zhang, 2024. "Double Descent in Portfolio Optimization: Dance between Theoretical Sharpe Ratio and Estimation Accuracy," Papers 2411.18830, arXiv.org.
    • Handle: RePEc:arx:papers:2411.18830
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2411.18830
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Tu, Jun & Zhou, Guofu, 2011. "Markowitz meets Talmud: A combination of sophisticated and naive diversification strategies," Journal of Financial Economics, Elsevier, vol. 99(1), pages 204-215, January.
    2. Stefano Giglio & Bryan Kelly & Dacheng Xiu, 2022. "Factor Models, Machine Learning, and Asset Pricing," Annual Review of Financial Economics, Annual Reviews, vol. 14(1), pages 337-368, November.
    3. Jianqing Fan & Yuan Liao & Martina Mincheva, 2013. "Large covariance estimation by thresholding principal orthogonal complements," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 75(4), pages 603-680, September.
    4. Shihao Gu & Bryan Kelly & Dacheng Xiu, 2020. "Empirical Asset Pricing via Machine Learning," Review of Finance, European Finance Association, vol. 33(5), pages 2223-2273.
    5. Victor DeMiguel & Lorenzo Garlappi & Raman Uppal, 2009. "Optimal Versus Naive Diversification: How Inefficient is the 1-N Portfolio Strategy?," The Review of Financial Studies, Society for Financial Studies, vol. 22(5), pages 1915-1953, May.
    6. Olivier Ledoit & Michael Wolf, 2017. "Nonlinear Shrinkage of the Covariance Matrix for Portfolio Selection: Markowitz Meets Goldilocks," The Review of Financial Studies, Society for Financial Studies, vol. 30(12), pages 4349-4388.
    7. Xi Dong & Yan Li & David E. Rapach & Guofu Zhou, 2022. "Anomalies and the Expected Market Return," Journal of Finance, American Finance Association, vol. 77(1), pages 639-681, February.
    8. Ravi Jagannathan & Tongshu Ma, 2003. "Risk Reduction in Large Portfolios: Why Imposing the Wrong Constraints Helps," Journal of Finance, American Finance Association, vol. 58(4), pages 1651-1683, August.
    9. Raymond Kan & Xiaolu Wang, 2024. "Optimal Portfolio Choice with Unknown Benchmark Efficiency," Management Science, INFORMS, vol. 70(9), pages 6117-6138, September.
    10. Joachim Freyberger & Andreas Neuhierl & Michael Weber & Andrew KarolyiEditor, 2020. "Dissecting Characteristics Nonparametrically," Review of Financial Studies, Society for Financial Studies, vol. 33(5), pages 2326-2377.
    11. Kan, Raymond & Zhou, Guofu, 2007. "Optimal Portfolio Choice with Parameter Uncertainty," Journal of Financial and Quantitative Analysis, Cambridge University Press, vol. 42(3), pages 621-656, September.
    12. Victor DeMiguel & Lorenzo Garlappi & Francisco J. Nogales & Raman Uppal, 2009. "A Generalized Approach to Portfolio Optimization: Improving Performance by Constraining Portfolio Norms," Management Science, INFORMS, vol. 55(5), pages 798-812, May.
    13. Michael W. Brandt & Pedro Santa-Clara & Rossen Valkanov, 2009. "Parametric Portfolio Policies: Exploiting Characteristics in the Cross-Section of Equity Returns," The Review of Financial Studies, Society for Financial Studies, vol. 22(9), pages 3411-3447, September.
    14. Ron Giammarino & Murray Carlson & Adlai Fisher, 2004. "Corporate Investment and Asset Price Dynamics: Implications for Post-SEO Performance," 2004 Meeting Papers 812, Society for Economic Dynamics.
    15. Victor DeMiguel & Alberto Martín-Utrera & Francisco J Nogales & Raman Uppal, 2020. "A Transaction-Cost Perspective on the Multitude of Firm Characteristics," The Review of Financial Studies, Society for Financial Studies, vol. 33(5), pages 2180-2222.
    16. Panos Xidonas & Ralph Steuer & Christis Hassapis, 2020. "Robust portfolio optimization: a categorized bibliographic review," Annals of Operations Research, Springer, vol. 292(1), pages 533-552, September.
    17. Jiaqin Chen & Ming Yuan, 2016. "Efficient Portfolio Selection in a Large Market," Journal of Financial Econometrics, Oxford University Press, vol. 14(3), pages 496-524.
    18. Shihao Gu & Bryan Kelly & Dacheng Xiu, 2020. "Empirical Asset Pricing via Machine Learning," The Review of Financial Studies, Society for Financial Studies, vol. 33(5), pages 2223-2273.
    19. Harry M. Markowitz, 2010. "Portfolio Theory: As I Still See It," Annual Review of Financial Economics, Annual Reviews, vol. 2(1), pages 1-23, December.
    20. repec:bla:jfinan:v:59:y:2004:i:6:p:2577-2603 is not listed on IDEAS
    21. Yuan Liao & Xinjie Ma & Andreas Neuhierl & Zhentao Shi, 2023. "Economic Forecasts Using Many Noises," Papers 2312.05593, arXiv.org, revised Dec 2023.
    22. Lorenzo Garlappi & Raman Uppal & Tan Wang, 2007. "Portfolio Selection with Parameter and Model Uncertainty: A Multi-Prior Approach," The Review of Financial Studies, Society for Financial Studies, vol. 20(1), pages 41-81, January.
    23. Jushan Bai, 2003. "Inferential Theory for Factor Models of Large Dimensions," Econometrica, Econometric Society, vol. 71(1), pages 135-171, January.
    24. Zhu Li & Weijie J. Su & Dino Sejdinovic, 2023. "Benign Overfitting and Noisy Features," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 118(544), pages 2876-2888, October.
    25. Silverstein, J. W., 1995. "Strong Convergence of the Empirical Distribution of Eigenvalues of Large Dimensional Random Matrices," Journal of Multivariate Analysis, Elsevier, vol. 55(2), pages 331-339, November.
    26. Doron Avramov & Si Cheng & Lior Metzker, 2023. "Machine Learning vs. Economic Restrictions: Evidence from Stock Return Predictability," Management Science, INFORMS, vol. 69(5), pages 2587-2619, May.
    27. Evan Anderson & Ai-ru (Meg) Cheng, 2022. "Portfolio Choices with Many Big Models," Management Science, INFORMS, vol. 68(1), pages 690-715, January.
    28. Raymond Kan & Xiaolu Wang & Guofu Zhou, 2022. "Optimal Portfolio Choice with Estimation Risk: No Risk-Free Asset Case," Management Science, INFORMS, vol. 68(3), pages 2047-2068, March.
    29. repec:bla:jfinan:v:58:y:2003:i:4:p:1651-1684 is not listed on IDEAS
    30. Bryan Kelly & Semyon Malamud & Kangying Zhou, 2024. "The Virtue of Complexity in Return Prediction," Journal of Finance, American Finance Association, vol. 79(1), pages 459-503, February.
    31. Stock J.H. & Watson M.W., 2002. "Forecasting Using Principal Components From a Large Number of Predictors," Journal of the American Statistical Association, American Statistical Association, vol. 97, pages 1167-1179, December.
    32. Victor DeMiguel & Alberto Martín-Utrera & Francisco J Nogales & Raman Uppal & Andrew KarolyiEditor, 2020. "A Transaction-Cost Perspective on the Multitude of Firm Characteristics," Review of Financial Studies, Society for Financial Studies, vol. 33(5), pages 2180-2222.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Tu, Xueyong & Li, Bin, 2024. "Robust portfolio selection with smart return prediction," Economic Modelling, Elsevier, vol. 135(C).
    2. Ni, Xuanming & Zheng, Tiantian & Zhao, Huimin & Zhu, Shushang, 2023. "High-dimensional portfolio optimization based on tree-structured factor model," Pacific-Basin Finance Journal, Elsevier, vol. 81(C).
    3. Ding, Yi & Li, Yingying & Zheng, Xinghua, 2021. "High dimensional minimum variance portfolio estimation under statistical factor models," Journal of Econometrics, Elsevier, vol. 222(1), pages 502-515.
    4. Francisco Peñaranda & Enrique Sentana, 2024. "Portfolio management with big data," Working Papers wp2024_2411, CEMFI.
    5. Thomas Conlon & John Cotter & Iason Kynigakis, 2021. "Machine Learning and Factor-Based Portfolio Optimization," Papers 2107.13866, arXiv.org.
    6. Johannes Bock, 2018. "An updated review of (sub-)optimal diversification models," Papers 1811.08255, arXiv.org.
    7. De Nard, Gianluca & Zhao, Zhao, 2023. "Using, taming or avoiding the factor zoo? A double-shrinkage estimator for covariance matrices," Journal of Empirical Finance, Elsevier, vol. 72(C), pages 23-35.
    8. Wolfgang Karl Hardle & Yegor Klochkov & Alla Petukhina & Nikita Zhivotovskiy, 2022. "Robustifying Markowitz," Papers 2212.13996, arXiv.org.
    9. Petukhina, Alla & Klochkov, Yegor & Härdle, Wolfgang Karl & Zhivotovskiy, Nikita, 2024. "Robustifying Markowitz," Journal of Econometrics, Elsevier, vol. 239(2).
    10. Chakrabarti, Deepayan, 2021. "Parameter-free robust optimization for the maximum-Sharpe portfolio problem," European Journal of Operational Research, Elsevier, vol. 293(1), pages 388-399.
    11. Härdle, Wolfgang & Klochkov, Yegor & Petukhina, Alla & Zhivotovskiy, Nikita, 2021. "Robustifying Markowitz," IRTG 1792 Discussion Papers 2021-018, Humboldt University of Berlin, International Research Training Group 1792 "High Dimensional Nonstationary Time Series".
    12. Fan, Jianqing & Liao, Yuan & Shi, Xiaofeng, 2015. "Risks of large portfolios," Journal of Econometrics, Elsevier, vol. 186(2), pages 367-387.
    13. Wang, Christina Dan & Chen, Zhao & Lian, Yimin & Chen, Min, 2022. "Asset selection based on high frequency Sharpe ratio," Journal of Econometrics, Elsevier, vol. 227(1), pages 168-188.
    14. Hautsch, Nikolaus & Voigt, Stefan, 2019. "Large-scale portfolio allocation under transaction costs and model uncertainty," Journal of Econometrics, Elsevier, vol. 212(1), pages 221-240.
    15. Hiraki, Kazuhiro & Sun, Chuanping, 2022. "A toolkit for exploiting contemporaneous stock correlations," Journal of Empirical Finance, Elsevier, vol. 65(C), pages 99-124.
    16. Chen, Jia & Li, Degui & Linton, Oliver, 2019. "A new semiparametric estimation approach for large dynamic covariance matrices with multiple conditioning variables," Journal of Econometrics, Elsevier, vol. 212(1), pages 155-176.
    17. N'Golo Kone, 2021. "Efficient mean-variance portfolio selection by double regularization," Working Paper 1453, Economics Department, Queen's University.
    18. Han, Chulwoo, 2020. "A nonparametric approach to portfolio shrinkage," Journal of Banking & Finance, Elsevier, vol. 120(C).
    19. Simon, Frederik & Weibels, Sebastian & Zimmermann, Tom, 2025. "Deep parametric portfolio policies," CFR Working Papers 23-01, University of Cologne, Centre for Financial Research (CFR), revised 2025.
    20. Sven Husmann & Antoniya Shivarova & Rick Steinert, 2022. "Sparsity and stability for minimum-variance portfolios," Risk Management, Palgrave Macmillan, vol. 24(3), pages 214-235, September.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2411.18830. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.