Abstract
A challenge for intelligent tutoring is to develop methodologies for transforming streams of performance data into insights and models about underlying learning mechanisms. Such modeling at different points in time could provide evidence of a student’s changing understanding of a task, and given sufficient detail, could extend our understanding of how gender, prior achievement, classroom practices and other student/contextual characteristics differentially influence performance and participation in complex problem-solving environments. If the models had predictive properties, they could also provide a framework for directing feedback to improve learning.
In this paper we describe the causal relationships between students’ problem-solving effectiveness (i.e. reaching a correct solution) and strategy (i.e. approach) and multiple contextual variables including experience, gender, classroom environment, and task difficulty. Performances of the IMMEX problem set Hazmat (n ~ 33,000) were first modeled by Item Response Theory analysis to provide a measure of effectiveness and then by self-organizing artificial neural networks and hidden Markov modeling to provide measures of strategic efficiency. Correlation findings were then used to link the variables into a Bayesian network representation. Sensitivity analysis indicated that whether a problem was solved or not was most likely influenced by findings related to the problem under investigation and the classroom environment while strategic approaches were most influenced by the actions taken, the classroom environment and the number of problems previously performed. Subsequent testing with unknown performances indicated that the strategic approaches were most easily predicted (17% error rate), whereas whether the problem was solved was more difficult (32% error rate).
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Arroyo, I., Woolf, B.: Inferring learning and attitudes from a Bayesian Network of log file data. In: Looi, C.-K., et al. (eds.) Artificial Intelligence in Education. IOS Press, Amsterdam (2005)
Alexander, P.A.: The Development of Expertise: The Journey from Acclimation to Proficiency. Educational Researcher 32(8), 10–14 (2003)
Brown, A., Palincsar, A.: Guided, Cooperative Learning and Individual Knowledge Acquisition. In: Resnick, L. (ed.) Knowing, learning, and instruction: Essays in honor of Robert Glaser. Lawrence Earlbaum Associates, NJ (1989)
Case, E.L.: The effects of collaborative grouping on problem solving in first-year chemistry. Ph.D. thesis, Clemson University (2004)
Chung, G.K.W.K., de Vries, L.F., Cheak, A.M., Stevens, R.H., Bewley, W.L.: Cognitive Process Validation of an Online Problem Solving Assessment. Computers and Human Behavior 18, 669 (2002)
Ericsson, K.A.: Deliberate Practice and the Acquisition and Maintenance of Expert Performance in Medicine and Related Domains. Academic Medicine 79(10), S70–S81 (2004)
Fennema, E., Carpenter, T., Jacobs, V., Franke, M., Levi, L.: Gender differences in mathematical thinking. Educational Researcher 27, 6–11 (1998)
Gallagher, A.M., De Lisi, R.: Gender differences in scholastic aptitude test mathematics problem solving among high ability students. Journal of Educational Psychology 86(2), 204–214 (1994)
Gerosa, L., Giordani, A., Ronchetti, M., Soller, A., Stevens, R.: Symmetric Synchronous Collaborative Navigation. In: A cura di Isaias, P., Karmakar, N. (eds.) IADIS International Conference WWW/Internet 2004, vol. 1, pp. 748–754. IADIS Press, Lisboa (2004)
Haider, H., Frensch, P.A.: The role of information reduction in skill acquisition. Cognitive Psychology 30, 304–337 (1996)
Hiebert, J., Stigler, J.: A Proposal for Improving Classroom Teaching: Lessons from the TIMSS Video Study. The Elementary School Journal 101, 3–21 (2000)
Kohonen, T.: Self Organizing Maps, 3rd extended edit. Springer, Heidelberg (2001)
Lajoie, S.P.: Transitions and Trajectories for Studies of Expertise. Educational Researcher 32, 21–25 (2003)
Lepper, M.R., Woolverton, M., Mumme, D., Gurtner, J.: Motivational Techniques of Expert Human Tutors: Lessons for the Design of Computer-Based Tutors. In: Lajoie, S.P., Derry, S.J. (eds.) Computers as cognitive tools, pp. 75–105. Erlbaum, Hillsdale (1993)
Linacre, J.M.: WINSTEPS Rasch measurement computer program. Chicago. Winsteps.com (2004)
Marshall, S.: Schemas in Problem Solving. Cambridge University Press, Cambridge (1995)
Mayer, R.E.: Should There Be A Three-Strikes Rule Against Pure Discovery Learning? American Psychologist 59, 14–19 (2004)
Mayer, R.E.: Cognitive, Metacognitive and Motivational Aspects of Problem Solving. Instructional Science 26, 49–63 (1998)
Mislevy, R.J., Steinberg, L.S., Breyer, F.J., Almond, R.G., Johnson, L.: A cognitive task analysis, with implications for designing a simulation-based assessment system. Computer and Human Behavior 15, 335–374 (1999)
Moreno, R., Duran, R.: Do Multiple Representations Need Explanations? The Role of Verbal Guidance and Individual Differences in Multimedia Mathematics Learning. Journal of Educational Psychology 96, 492–503 (2004)
Murphy, K.: Hidden Markov Model (HMM) Toolbox for Matlab (2004), Available online at: http://www.cs.ubc.ca/~murphyk/Software/HMM/hmm.html
O’Regan, K.: Emotion and e-Learning. Journal of Asynchronous Learning Network 7(3), 78–92 (2003)
Olson, A., Loucks-Horsley, S. (eds.): Inquiry and the National Science Education Standards: A guide for teaching and learning. National Academy Press, Washington (2000)
Rabiner, L.: A tutorial on Hidden Markov Models and selected applications in speech recognition. Proceedings of IEEE 77, 257–286 (1989)
Schunn, C.D., Anderson, J.R.: The Generality/Specificity of Expertise in Scientific Reasoning. Cognitive Science 13, 337–370 (1999)
Soller, A., Lesgold, A.: A Computational Approach to Analyzing Online Knowledge Sharing Interaction. In: Proceedings of Artificial Intelligence in Education, Sydney Australia, pp. 253–260 (2003)
Soller, A., Wiebe, J., Lesgold, A.: A Machine Learning Approach to Assessing Knowledge Sharing During Collaborative Learning Activities. In: Proceedings of Computer Support for Collaborative Learning, Boulder, CO, pp. 128–137 (2002)
Stevens, R.H., Palacio-Cayetano, J.: Design and Performance Frameworks for Constructing Problem-solving Simulations. Cell Biology Education 2, 162–179 (2003)
Stevens, R.H., Ikeda, J., Casillas, A., Palacio-Cayetano, J., Clyman, S.: Artificial Neural Network-based Performance Assessments. Computers in Human Behavior 15, 295–314 (1999)
Stevens, R.H., Wang, P., Lopo, A.: Artificial Neural Networks Can Distinguish Novice and Expert Strategies During Complex Problem Solving. JAMIA 3(2), 131–138 (1996)
Stevens, R., Johnson, D.F., Soller, A.: Probabilities and Predictions: Modeling the Development of Scientific Competence. Cell Biology Education, The American Society for Cell Biology 4, 42–57 (2005)
Stevens, R., Soller, A., Cooper, M., Sprang, M.: Modeling the Development of Problem Solving Skills in Chemistry with a Web-Based Tutor. In: Lester, J.C., Vicari, R.M., Paraguaçu, F. (eds.) ITS 2004. LNCS, vol. 3220, pp. 580–591. Springer, Heidelberg (2004)
Stevens, R.H., Najafi, K.: Artificial Neural Networks as Adjuncts for Assessing Medical Students’ Problem-solving Performances on Computer-based Simulations. Computers and Biomedical Research 26, 172–187 (1993)
Stevens, R.H., Ikeda, J., Casillas, A., Palacio-Cayetano, J., Clyman, S.: Artificial neural network-based performance assessments. Computers in Human Behavior 15, 295–314 (1999)
Stevens, R.H., Soller.: Implementing a Layered Analytic Approach for Real-Time Modeling of Student’s Scientific Understanding. In: Proceedings of the Artificial Intelligence in Education Conference (2005)
Sugrue, B.: A theory-based framework for assessing domain-specific problem solving ability. Educational Measurement: Issues and Practice 14(3), 29–36 (1995)
Topping, K.J., Ehly, S. (eds.): Peer-assisted learning. Lawrence Erlbaum, Mahwah (1998)
Underdahl, J., Palacio-Cayetano, J., Stevens, R.: Practice makes perfect: assessing and enhancing knowledge and problem-solving skills with IMMEX software. Learning and Leading with Technology 28, 26–31 (2001)
VanLehn, K.: Cognitive Skill Acquisition. Annual Review. Psychology. 47, 513–539 (1996)
VanLehn, K., Ohlsson, S., Nason, R.: Applications of Simulated Students: An Exploration. Journal of Artificial Intelligence in Education 5(2), 135–175 (1994)
Vendlinski, T., Stevens, R.: A Markov Model Analysis of Problem-Solving Progress and Transfer. Journal of Technology, Learning, and Assessment 1(3) (2002)
Webb, N.: Testing a Theoretical Model of Student Interaction and Learning in Small Groups. In: Hertz-Lazarowitz, R., Miller, N. (eds.) Interaction in Cooperative Groups: The Theoretical Anatomy of Group Learning, pp. 102–119. Cambridge University Press, New York (1992)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Stevens, R.H., Thadani, V. (2006). A Bayesian Network Approach for Modeling the Influence of Contextual Variables on Scientific Problem Solving. In: Ikeda, M., Ashley, K.D., Chan, TW. (eds) Intelligent Tutoring Systems. ITS 2006. Lecture Notes in Computer Science, vol 4053. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11774303_8
Download citation
DOI: https://doi.org/10.1007/11774303_8
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-35159-7
Online ISBN: 978-3-540-35160-3
eBook Packages: Computer ScienceComputer Science (R0)