Abstract
Recent developments in combined human-computer tutoring systems show promise in narrowing math achievement gaps among marginalized students. We present an evaluation of the use of the Personalized Learning2, a hybrid tutoring approach whereby human mentoring and AI tutoring are combined to personalize learning with respect to students’ motivational and cognitive needs. The approach assumes achievement gaps emerge from differences in learning opportunities and seeks to increase such opportunities for marginalized students through after-school programs, such as the Ready to Learn program. This program engaged diverse middle school students from three schools in an urban district. We compared achievement growth of 70 treatment students in this program with a control group of 380 students from the same district selected by propensity matching to have similar demographics and prior achievement. Based on standardized math assessments (NWEA Measures of Academic Progress) given one year apart, we found the gain of treatment students (6.8 points) was nearly double the gain of the control group (3.6 points). Further supporting the inference that greater learning was caused by the math-focused treatment and not by some selection bias, we found no significant differences in reading achievement between treatment and control participants. These results show promise that greater educational equity can be achieved at reasonable costs through after-school programs that combine the use of low-cost paraprofessional mentors and computer-based tutoring.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
- 2.
Estimated mentor cost: $15/hr.*2 h./session*2 sessions/week*24 weeks/4 students per mentor = $360 per student/year.
References
Alexander, K., Pitcock, S., Boulay, M.C. (eds.): The Summer Slide: What We Know and Can Do About Summer Learning Loss. Teachers College Press (2016)
Autor, D.: Skills, education, and the rise of earnings inequality among the “other 99 percent.” Science 344(6186), 843–851 (2014)
Cook, P.: Not too late: improving academic outcomes for disadvantaged youth. Northwestern University Institute for Policy Research. Working Paper No. 15-01 (2015)
Duckworth, A.L., Kirby, T.A., Tsukayama, E., Berstein, H., Ericsson, K.A.: Deliberate practice spells success: why grittier competitors triumph at the National Spelling Bee. Soc. Psychol. Pers. Sci. 2(2), 174–181 (2011)
Ericsson, K.A., Krampe, R.T., Tesch-Römer, C.: The role of deliberate practice in the acquisition of expert performance. Psychol. Rev. 100(3), 363 (1993)
Graesser, A.C., Forsyth, C.M., Lehman, B.A.: Two heads may be better than one: learning from computer agents in conversational trialogues. Teachers Coll. Board 119(3), 1–20 (2017)
Guryan, J.: The effect of mentoring on school attendance and academic outcomes: a randomized evaluation of the check & connect program. Institute for Policy Research Working Paper Series, WP-16-18. Northwestern University (2017)
Guryan, J.: Not too late: improving academic outcomes among adolescents (No. w28531). National Bureau of Economic Research (2021)
Hansen, B.B.: Full matching in an observational study of coaching for the SAT. J. Am. Stat. Assoc. 99(1), 609–619 (2004)
Harackiewicz, J., Rozek, C., Hulleman, C., Hyde, J.: Helping parents to motivate adolescents in mathematics and science: an experimental test of a utility-value intervention. Psychol. Sci. 23(8), 899–906 (2012)
Heller, S.B., Shah, A.K., Guryan, J., Ludwig, J., Mullainathan, S., Pollack, H.A.: Thinking, fast and slow? Some field experiments to reduce crime and dropout in Chicago. Q. J. Econ. 132(1), 1–54 (2017)
Hutson, K.M.: Missing faces: making the case for equitable student representation in advanced middle school courses. In: Promoting Positive Learning Experiences in Middle School Education, pp. 200–216. IGI Global (2021)
Kim, K.H., Zabelina, D.: Cultural bias in assessment: can creativity assessment help? Int. J. Crit. Pedagogy 6(2), 129–148 (2015)
Kumar, R., Rosé, C.P., Wang, Y.C., Joshi, M., Robinson, A.: Tutorial dialogue as adaptive collaborative learning support. Front. Artif. Intell. Appl. 158, 383 (2007)
Lewis, K., Kuhfeld, M.: Learning during COVID-19: an update on student achievement and growth at the start of the 2021–22 school year. NWEA (2021)
Lewis, K., Kuhfeld, M., Ruzek, E., McEachin, A.: Learning during COVID-19: reading and math achievement in the 2020–21 school year. NWEA (2021)
Matayoshi, J., Uzun, H., Cosyn, E.: Studying retrieval practice in an intelligent tutoring system. In: Proceedings of the Seventh ACM Conference on Learning@ Scale, pp. 51–62. Springer, August 2020
Milner, H.R.: Start Where You Are, But Don’t Stay There: Understanding Diversity, Opportunity Gaps, and Teaching in Today’s Classrooms, 2nd edn. Harvard Education Press, Cambridge (2020)
Muralidharan, K., Singh, A., Ganimian, A.J.: Disrupting education? Experimental evidence on technology-aided instruction in India. Am. Econ. Rev. 109(4), 1426–1460 (2019)
NWEA. 2020 MAP growth normative data overview, March 2020. https://teach.mapnwea.org/impl/MAPGrowthNormativeDataOverview.pdf
Pane, J.F., Griffin, B.A., McCaffrey, D.F., Karam, R.: Effectiveness of cognitive tutor algebra I at scale. Educ. Eval. Policy Anal. 36(2), 127–144 (2014)
Popham, W.J.: Assessment Bias: How to Banish It. Routledge, New York (2006)
Ritter, S., Anderson, J.R., Koedinger, K.R., Corbett, A.: Cognitive tutor: applied research in mathematics education. Psychon. Bull. Rev. 14(2), 249–255 (2007). https://doi.org/10.3758/BF03194060
Schaldenbrand, P.: Computer-supported human mentoring for personalized and equitable math learning. In: Roll, I., McNamara, D., Sosnovsky, S., Luckin, R., Dimitrova, V. (eds.) International Conference on Artificial Intelligence in Education. AIED 2021. LNCS, vol. 12749, pp. 308–313. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-78270-2_55
Setren, E.M.: Essays on the economics of education. (Unpublished doctoral dissertation). Massachusetts Institute of Technology (2017)
Siegle, D., McCoach, D.B.: Increasing student mathematics self-efficacy through teacher training. J. Adv. Acad. 18(2), 278–312 (2007)
Stuart, E.A., Green, K.M.: Using full matching to estimate causal effects in nonexperimental studies: examining the relationship between adolescent marijuana use and adult outcomes. Dev. Psychol. 44(2), 395–406 (2008). https://doi.org/10.1037/0012-1649.44.2.395
Trochim, W.M., Donnelly, J.P.: The Research Methods Knowledge Base, 3rd edn. Cengage, Ohio (2007)
Van Buuren, S., Groothuis-Oudshoorn, K.: MICE: multivariate imputation by chained equations in R. J. Stat. Softw. 45, 1–67 (2011)
Walker, E.: Automated adaptive support for peer tutoring. Doctoral dissertation, Carnegie Mellon University (2011)
Walton, G.M., Cohen, G.L.: A brief social-belonging intervention improves academic and health outcomes of minority students. Science 331(6023), 1447–1451 (2011)
Yeager, D.S., Dweck, C.S.: Mindsets that promote resilience: when students believe that personal characteristics can be developed. Educ. Psychol. 47(4), 302–314 (2012)
Acknowledgements
This work is supported with funding from the Chan Zuckerberg Initiative (Grant #2018-193694), Richard King Mellon Foundation (Grant #10851), Bill and Melinda Gates Foundation, and the Heinz Endowments (E6291). Any opinions and conclusions expressed in this material are those of the authors
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 Springer Nature Switzerland AG
About this paper
Cite this paper
Chine, D.R. et al. (2022). Educational Equity Through Combined Human-AI Personalization: A Propensity Matching Evaluation. In: Rodrigo, M.M., Matsuda, N., Cristea, A.I., Dimitrova, V. (eds) Artificial Intelligence in Education. AIED 2022. Lecture Notes in Computer Science, vol 13355. Springer, Cham. https://doi.org/10.1007/978-3-031-11644-5_30
Download citation
DOI: https://doi.org/10.1007/978-3-031-11644-5_30
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-11643-8
Online ISBN: 978-3-031-11644-5
eBook Packages: Computer ScienceComputer Science (R0)