research-article

Public Access

Predicting Individual Differences for Learner Modeling in Intelligent Tutors from Previous Learner Activities

Authors:

Albert Corbett,

Bruce M. McLaren,

Benjamin MacLaren,

Aaron MitchellAuthors Info & Claims

UMAP '16: Proceedings of the 2016 Conference on User Modeling Adaptation and Personalization

Pages 55 - 63

https://doi.org/10.1145/2930238.2930255

Published: 13 July 2016 Publication History

Abstract

This study examines how accurately individual student differences in learning can be predicted from prior student learning activities. Bayesian Knowledge Tracing (BKT) predicts learner performance well and has often been employed to implement cognitive mastery. Standard BKT individualizes parameter estimates for knowledge components, but not for learners. Studies have shown that individualizing parameters for learners improves the quality of BKT fits and can lead to very different (and potentially better) practice recommendations. These studies typically derive best-fitting individualized learner parameters from learner performance in existing data logs, making the methods difficult to deploy in actual tutor use. In this work, we examine how well BKT parameters in a tutor lesson can be individualized based on learners' prior performance in reading instructional text, taking a pretest, and completing an earlier tutor lesson. We find that best-fitting individual difference estimates do not directly transfer well from one tutor lesson to another, but that predictive models incorporating variables extracted from prior reading, pretest and tutor activities perform well, when compared to a standard BKT model and a model with best-fitting individualized parameter estimates.

References

[1]

Corbett, A.T., Anderson, J.R.: Knowledge tracing: Modeling the acquisition of procedural knowledge. User modeling and user-adapted interaction, 4, 253--278. (1995)

[2]

Mayo, M., Mitrovic, A. Optimising ITS behaviour with Bayesian networks and decision theory. International Journal of Artificial Intelligence in Education, 12, 124--153 (2001).

[3]

Shute, V.: Smart: Student Modeling Approach for Responsive Tutoring. User Modeling and User-Adapted Interaction, 5 (1), 1--44. (1995)

[4]

Ganeshan, R., Johnson, L., Shaw, E., Wood, B.: Tutoring diagnostic problem solving. In G. Gauthier, C. Frasson, K. VanLehn (eds.) ITS2000 Intelligent Tutoring Systems, LNCS vol. 1839, pp. 33--42. Springer, Heidelberg. (2000)

Digital Library

[5]

Conati, C., Gertner, A., VanLehn, K. Using Bayesian networks to manage uncertainty in student modeling. User Modeling and User-Adapted Interaction, 12, 371--417. (2002)

Digital Library

[6]

Corbett, A.T., MacLaren, B., Kauffman, L., Wagner, A., Jones, E. A.: Cognitive Tutor for Genetics Problem Solving: Learning Gains and Student Modeling. Journal of Educational Computing Research, 42 (2), 219--239. (2010)

[7]

Gong, Y., Beck, J., Heffernan, N.: Comparing knowledge tracing and performance factor analysis by using multiple model fitting. In V. Aleven, J. Kay, J. Mostow (eds.) ITS2010 Intelligent Tutoring Systems. LNCS vol. 6094, pp. 35--44. Springer, Heidelberg. (2010)

Digital Library

[8]

Lee, J., Brunskill, E.: The impact of individualizing student models on necessary practice opportunities. In: Yacef, K., Zaiane, O., Hershkovitz, A., Yudelson, M., Stamper, J. (eds.) EDM2012 Proceedings of the 5th International Conference on International Educational Data Mining Society, 118--125. (2012)

[9]

Pardos, Z. Heffernan, N.: Modeling individualization in a Bayesian networks implementation of Knowledge Tracing. In De Bra, P., Kobsa, A., Chin, D. (eds.) UMAP2010 Proceedings of the 18th International Conference on User Modeling, Adaptation, and Personalization, LNCS vol. 6075, pp. 255--266. Springer, Heidelberg (2010)

Digital Library

[10]

Yudelson, M., Koedinger, K., Gordon, G.: Individualized Bayesian knowledge tracing models. In: Lane, C., Yacef, K., Mostow, J., Pavlik, P. (eds.) AIED2013 Artificial Intelligence in Education, LNCS vol. 7926, pp. 171--180, Springer, Heidel-berg. (2013)

[11]

Pirolli, P., Wilson, M: A theory of the measurement of knowledge content, access, and learning. Psychological Review, 105(1), 58--82. (1998)

[12]

Cen, H., Koedinger, K., Junker, B.: Comparing two IRT models for conjunctive skills. In B. Woolf, E. Atmour, R. Nkambou, S. Lajoie (eds.) ITS2008 Intelligent Tutoring Systems LNCS vol. 5091, pp. 796--798. Springer, Heidelberg. (2008)

Digital Library

[13]

Pavlik, P., Yudelson, M., Koedinger, K.: Using contextual factors analysis to explain transfer of least common multiple skills. In G. Biswas, S. Bull, J. Kay, A. Mitrovic (eds.) AIED2011 Artificial Intelligence in Education, LNCS vol. 6738, pp. 256--263. Springer, Heidelberg. (2011)

Digital Library

[14]

Eagle, M., Corbett, A., Stamper, J., McLaren, B.M., Wagner, A., MacLaren, B., Mitchell, A.: Estimating individual differences for student modeling in intelligent tutors from reading and pretest data. ITS2016 Intelligent Tutoring Systems. (in press)

Digital Library

[15]

Harvey, L., Anderson, J.: Transfer of declarative knowledge in complex information processing domains. Human-Computer Interaction, 11 (1), 69--96. (1996)

Digital Library

[16]

Zwann, R., Singer, M.: Text comprehension. In A. Graesser, M. Gernsbacher, S. Goldman (eds.) Handbook of discourse processes, pp. 83--12 Mahwah, NJ: Erlbaum. (2003)

[17]

Arroyo, I., Beck, J., Woolf, B., Beal, C., Schultz, K.: Macroadapting Animalwatch to gender and cognitive differences with respect to hint interactivity and symbolism. In G. Gauthier, C. Frasson, K. VanLehn (eds.) ITS2000 Intelligent Tutoring Systems, LNCS vol. 1839, pp. 574--583. Springer, Heidelberg. (2000)

Digital Library

[18]

Bransford, J. D., & Schwartz, D. (1999). Rethinking transfer: A simple proposal with multiple implications. In A. Iran-Nejad & P. D. Pearson (Eds.), Review of Research in Education (Vol. 24). Washington, DC: American Educational Research Association.

[19]

Baker, R.S.J.d., Corbett, A.T., Gowda, S.M. (2013). Generalizing automated detection of the robustness of student learning in an intelligent tutor for genetics. Journal of Educational Psychology. 105, 946--956.

[20]

Koedinger, K.R., Corbett, A.T. and Perfetti, C. (2012). The Knowledge-Learning-Instruction (KLI) Framework: Bridging the Science-Practice Chasm to Enhance Robust Student Learning. Cognitive Science, 36, 757--798.

[21]

Baker, R.S.J.d., Gowda, S.M., Corbett, A.T., Ocumpaugh, J.: Towards automatically detecting whether student learning is shallow. In S. Cerri, W. Clancey, G. Papadourakis, K. Panourgia (eds.) ITS2012 Intelligent Tutoring Systems LNCS vol. 7315, pp. 444--453. Springer, Heidelberg. (2012)

Digital Library

[22]

Koedinger, K., Stamper, J., McLaughlin, E., Nixon,T.: Using data-driven discovery of better student models to improve student learning. In: Lane, C., Yacef, K., Mostow, J., Pavlik, P. (eds.) AIED2013 Artificial Intelligence in Education, LNCS vol. 7926, pp. 421--430, Springer, Heidelberg. (2013)

[23]

Baker, R., Corbett, A., Aleven, V.: More Accurate Student Modeling Through Contextual Estimation of Slip and Guess Probabilities in Bayesian Knowledge Tracing. In B. Woolf, E. Aimeur, R. Nkambou, S. Lajoie (eds.) ITS2008 Intelligent Tutoring Systems. LNCS vol. 5091, pp. 406--415. Springer, Heidelberg. (2008)

Digital Library

[24]

Efron, B., Hastie, T., Johnstone, I., & Tibshirani, R. Least angle regression. The Annals of statistics, 32(2), 407--499. (2004).

[25]

Andersen, Robert. Modern methods for robust regression. Sage, 2008.

[26]

Ihaka, Ross, and Robert Gentleman. "R: a language for data analysis and graphics." Journal of computational and graphical statistics 5.3, 299--314. (1996)

[27]

Nash, John C., and Ravi Varadhan. "Unifying optimization algorithms to aid software system users: optimx for R." Journal of Statistical Software 43.9, 1--14. (2011)

[28]

Venables, W. N., Ripley, B.D., Modern Applied Statistics with S. Forth Edition. Springer, New York. (2013)

Digital Library

Cited By

Ines ŠAni GAngelina G(2024)Twenty-Five Years of Bayesian knowledge tracing: a systematic reviewUser Modeling and User-Adapted Interaction10.1007/s11257-023-09389-4Online publication date: 27-Jan-2024
https://doi.org/10.1007/s11257-023-09389-4
Cao JGanesh ACai JSouthwell RPerkoff ERegan MKann KMartin JPalmer MD'Mello S(2023)A Comparative Analysis of Automatic Speech Recognition Errors in Small Group Classroom DiscourseProceedings of the 31st ACM Conference on User Modeling, Adaptation and Personalization10.1145/3565472.3595606(250-262)Online publication date: 18-Jun-2023
https://dl.acm.org/doi/10.1145/3565472.3595606
Hou XNguyen HRichey JHarpstead EHammer JMcLaren B(2021)Assessing the Effects of Open Models of Learning and Enjoyment in a Digital Learning GameInternational Journal of Artificial Intelligence in Education10.1007/s40593-021-00250-632:1(120-150)Online publication date: 13-Apr-2021
https://doi.org/10.1007/s40593-021-00250-6
Show More Cited By

Index Terms

Predicting Individual Differences for Learner Modeling in Intelligent Tutors from Previous Learner Activities
1. Applied computing
  1. Education
    1. Computer-managed instruction
    2. Interactive learning environments
2. Human-centered computing
  1. Human computer interaction (HCI)
    1. HCI design and evaluation methods
      1. User models

Recommendations

Estimating Individual Differences for Student Modeling in Intelligent Tutors from Reading and Pretest Data
ITS 2016: Proceedings of the 13th International Conference on Intelligent Tutoring Systems - Volume 9684

Past studies have shown that Bayesian Knowledge Tracing BKT can predict student performance and implement Cognitive Mastery successfully. Standard BKT individualizes parameter estimates for skills, also referred to as knowledge components KCs, but not ...
Using Big Data and BKT to Evaluate Course Resources (Abstract Only)
SIGCSE '15: Proceedings of the 46th ACM Technical Symposium on Computer Science Education

Now finding footing in objective research methodology, MOOCs have made significant strides toward developing into mature platforms for not only offering educational materials but also performing exploratory analysis of educational methods. Bayesian ...
Scrutable learner modelling and learner reflection in student self-assessment
ITS'06: Proceedings of the 8th international conference on Intelligent Tutoring Systems

Learner reflection is critical to effective, deep, transferable learning, especially in cognitively demanding areas, such as learning programming. This paper presents KrAssess, a programming education system, which aims to facilitate student self-...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

UMAP '16: Proceedings of the 2016 Conference on User Modeling Adaptation and Personalization

July 2016

366 pages

ISBN:9781450343688

DOI:10.1145/2930238

General Chairs:
Julita Vassileva
University of Saskatchewan, Canada
,
James Blustein
Dalhousie University, Canada
,
Program Chairs:
Lora Aroyo
VU Amsterdam, the Netherlands
,
Sidney D'Mello
University of Notre Dame, USA

Copyright © 2016 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 13 July 2016

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

National Science Foundation

Conference

UMAP '16

Sponsor:

UMAP '16: User Modeling, Adaptation and Personalization Conference

July 13 - 17, 2016

Nova Scotia, Halifax, Canada

Acceptance Rates

UMAP '16 Paper Acceptance Rate 21 of 123 submissions, 17%;

Overall Acceptance Rate 162 of 633 submissions, 26%

Upcoming Conference

UMAP '25

Sponsor:
sigchi
sigchi

33rd ACM Conference on User Modeling, Adaptation and Personalization

June 16 - 19, 2025

New York City , NY , USA

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
621
Total Downloads

Downloads (Last 12 months)42
Downloads (Last 6 weeks)10

Reflects downloads up to 04 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Ines ŠAni GAngelina G(2024)Twenty-Five Years of Bayesian knowledge tracing: a systematic reviewUser Modeling and User-Adapted Interaction10.1007/s11257-023-09389-4Online publication date: 27-Jan-2024
https://doi.org/10.1007/s11257-023-09389-4
Cao JGanesh ACai JSouthwell RPerkoff ERegan MKann KMartin JPalmer MD'Mello S(2023)A Comparative Analysis of Automatic Speech Recognition Errors in Small Group Classroom DiscourseProceedings of the 31st ACM Conference on User Modeling, Adaptation and Personalization10.1145/3565472.3595606(250-262)Online publication date: 18-Jun-2023
https://dl.acm.org/doi/10.1145/3565472.3595606
Hou XNguyen HRichey JHarpstead EHammer JMcLaren B(2021)Assessing the Effects of Open Models of Learning and Enjoyment in a Digital Learning GameInternational Journal of Artificial Intelligence in Education10.1007/s40593-021-00250-632:1(120-150)Online publication date: 13-Apr-2021
https://doi.org/10.1007/s40593-021-00250-6
Baker RMcLaren BHutt SRichey JRowe EAlmeda MMogessie MAndres J(2021)Towards Sharing Student Models Across Learning SystemsArtificial Intelligence in Education10.1007/978-3-030-78270-2_10(60-65)Online publication date: 12-Jun-2021
https://doi.org/10.1007/978-3-030-78270-2_10
Han JJo JJi HLim H(2016)A collaborative recommender system for learning courses considering the relevance of a learner's learning skillsCluster Computing10.1007/s10586-016-0670-x19:4(2273-2284)Online publication date: 1-Dec-2016
https://dl.acm.org/doi/10.1007/s10586-016-0670-x
Baneres D(2016)Towards a Particular Prediction System to Evaluate Student’s SuccessAdvances on P2P, Parallel, Grid, Cloud and Internet Computing10.1007/978-3-319-49109-7_91(935-945)Online publication date: 22-Oct-2016
https://doi.org/10.1007/978-3-319-49109-7_91

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents