[go: up one dir, main page]

Skip to main content

Observing Lemmatization Effect in LSA Coherence and Comprehension Grading of Learner Summaries

  • Conference paper
Intelligent Tutoring Systems (ITS 2006)

Part of the book series: Lecture Notes in Computer Science ((LNPSE,volume 4053))

Included in the following conference series:

Abstract

Current work in learner evaluation of Intelligent Tutoring Systems (ITSs), is moving towards open-ended educational content diagnosis. One of the main difficulties of this approach is to be able to automatically understand natural language. Our work is directed to produce automatic evaluation of learner summaries in Basque. Therefore, in addition to language comprehension, difficulties emerge from Basque morphology itself. In this work, Latent Semantic Analysis (LSA) is used to model comprehension in a language in which lemmatization has shown to be highly significant. This paper tests the influence of corpus lemmatization while performing automatic comprehension and coherence grading. Summaries graded by human judges in coherence and comprehension, have been tested against LSA based measures from source lemmatized and non-lemmatized corpora. After lemmatization, the amount of LSA known single terms was reduced in a 56% of its original number. As a result, LSA grades almost match human measures, producing no significant differences between the lemmatized and non-lemmatized approaches.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Similar content being viewed by others

References

  1. Kintsch, W., Patel, V.L., Ericsson, K.A.: The role of long-term working memory in text comprehension. Psychologia 42, 186–198 (1999)

    Google Scholar 

  2. Barlett, F.C.: Remembering; a Studty in Experimental and Social Psychology. Cambridge University Press, Cambridge (1932)

    Google Scholar 

  3. Garner, R.: Efficient Text Summarization. Costs and Benefits. Journal of Educational Research 75(5), 275–279 (1982)

    Google Scholar 

  4. Zipitria, I., Elorriaga, J.A., Arruarte, A., de Ilarraza, A.D.: From Human to Automatic Summary Evaluation. In: Lester, J.C., Vicari, R.M., Paraguaçu, F. (eds.) ITS 2004. LNCS, vol. 3220, pp. 432–442. Springer, Heidelberg (2004)

    Chapter  Google Scholar 

  5. Landauer, T.K., Dumais, S.T.: A solution to Plato’s problem: The Latent Semantic Analysis theory of acquisition, induction, and representation of knowledge. Psychological Review 104, 211–240 (1997)

    Article  Google Scholar 

  6. Deerwester, S., Dumais, S.T., Furnas, G.W., Landauer, T.K., Harshman, R.: Indexing by Latent Semantic Analysis. Journal of the American Society of Information Science (1990)

    Google Scholar 

  7. Landauer, T.K., Foltz, P., Laham, D.: Introduction to Latent Semantic Analysis. Discourse Processes 25, 259–284 (1998)

    Article  Google Scholar 

  8. Foltz, P.W., Kintsch, W., Landauer, T.K.: The Measurement of Textual Coherence with Latent Semantic Analysis. Discourse Processes 25, 285–307 (1998)

    Article  Google Scholar 

  9. Wolfe, M.B.W., Schreiner, M.E., Rehder, B., Laham, D., Foltz, P.W., Kintsch, W., Lan-dauer, T.K.: Learning from text:Matching readers and texts by Latent Semantic Analysis. Discourse Processes 25, 309–336 (1998)

    Article  Google Scholar 

  10. Graesser, A.C., Person, N.K., Harter, D.: Teaching tactics and dialog in Autotutor. International Journal of Artificial Intelligence in Education 12, 257–279 (2001)

    Google Scholar 

  11. Wiemer-Hastings, P., Graesser, A.: Select-a-Kibitzer: A computer tool that gives meaningful feedback on student compositions. Interactive Learning Environments 8(2), 149–169 (2000)

    Article  Google Scholar 

  12. Wade-Stein, D., Kintsch, E.: Summary Street: Interactive Computer Support for Writing. Cognition and Instruction 22(3), 333–362 (2004)

    Article  Google Scholar 

  13. Miller, T.: Essay assessment with latent semantic analysis. Journal of Educational Computing Research 28 (2003)

    Google Scholar 

  14. Ventura, M.J., Franchescetti, D.R., Pennumatsa, P., Graesser, A.C., Hu, G.T.J.X., Cai, Z., Group, t.T.R.: Combining Computational Models of Short Essay Grading for Conceptual Physics Problems. In: Lester, J.C., Vicari, R.M., Paraguaçu, F. (eds.) ITS 2004. LNCS, vol. 3220, pp. 423–431. Springer, Heidelberg (2004)

    Chapter  Google Scholar 

  15. Tomasello, M.: Constructing a Language: A Usage-Based Theory of Language Acquisition. Harvard University Press, Cambridge (2003)

    Google Scholar 

  16. Palolahti, M., Leino, S., Jokela, M., Kopra, K., Paavilainen, P.: Event-related potentials suggest early interaction between syntax and semantics during on-line sentence comprehension. Neuroscience Letters 384(3), 222 (2005)

    Article  Google Scholar 

  17. Hagoort, P.: Interplay between Syntax and Semantics during Sentence Comprehension: ERP Effects of Combining Syntactic and Semantic Violations. Journal of Cognitive Neuroscience 15(6), 883–899 (2003)

    Article  Google Scholar 

  18. Landauer, T.K., Laham, D., Rehder, B., Schreiner, M.E.: How well can passage meaning be derived without using word order? A comparison of Latent Semantic Analysis and humans. In: 19th Annual Meeting of the Cognitive Science Society. Erlbaum, Mahwah (1997)

    Google Scholar 

  19. Wiemer-Hastings, P., Zipitria, I.: Rules for Syntax, Vectors for Semantics. In: Proceedings of the 23rd Annual Conference of the Cognitive Science Society. Erlbaum, Mahwah (2001)

    Google Scholar 

  20. Serafin, R., Eugenio, B.D.: FLSA: Extending Latent Semantic Analysis with Features for Dialogue Act Classification. In: Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics, Barcelona, Spain (2004)

    Google Scholar 

  21. Kanejiya, D., Kamar, A., Prasad, S.: Automatic Evaluation of Students’ Answers using Syntactically Enhanced LSA. In: Proceedings of the HLT-NAACL 2003 Workshop on Building Educational Applications Using Natural Language Processing (2003)

    Google Scholar 

  22. Olde, B.A., Franceschetti, D.R., Karnavat, A., Graesser, A.C., TRG.: The right stuff: Do you need to sanitize your corpus when using latent semantic analysis? In: 24rd Annual Conference of the Cognitive Science Society. Erlbaum, Mahwah (2002)

    Google Scholar 

  23. Landauer, T.K., Littman, M.L.: A statistical method for language-independent representation of the topical content of text segments. In: Proceedings of the Sixth Annual Conference of the UW Centre for the New Oxford English Dictionary and Text Research (1990)

    Google Scholar 

  24. Aduriz, I., Aranzabe, M.J., Arriola, J.M., de Ilarraza, A.D., Gojenola, K., Oronoz, M., Uria, L.: A Cascaded Syntactic Analyser for Basque. In: Gelbukh, A. (ed.) CICLing 2004. LNCS, vol. 2945, pp. 124–134. Springer, Heidelberg (2004)

    Chapter  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2006 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Zipitria, I., Arruarte, A., Elorriaga, J.A. (2006). Observing Lemmatization Effect in LSA Coherence and Comprehension Grading of Learner Summaries. In: Ikeda, M., Ashley, K.D., Chan, TW. (eds) Intelligent Tutoring Systems. ITS 2006. Lecture Notes in Computer Science, vol 4053. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11774303_59

Download citation

  • DOI: https://doi.org/10.1007/11774303_59

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-35159-7

  • Online ISBN: 978-3-540-35160-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics