Abstract
Documents are composed of words, and machine learning methods process numerical vectors. This chapter discusses how words are transformed into vectors, readying them for processing by predictive methods. Documents may appear in different formats and may be collected from different sources. With minor modifications, they can be organized and unified for prediction by specifying them in a standard descriptive language, XML. The words or tokens may be further reduced to common roots by stemming. These tokens are added to a dictionary. The words in a document can be converted to vectors using local or global dictionaries. The value of each entry in the vector will be based on measures of frequency of occurrence of words in a document such as term frequency (tf and idf). An additional entry in a document vector is a label of the correct answer, such as its topic. Dictionaries can be extended to multiword features like phrases. Dictionary size may be significantly reduced by attribute ranking. The general approach is purely empirical, preparing data for statistical prediction. Linguistic concepts are also discussed including part-of-speech tagging, word sense disambiguation, phrase recognition, parsing and feature generation.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
C. Apté, F. Damerau, and S. Weiss. Automated learning of decision rules for text categorization. ACM Transactions on Information Systems, 12(3):233–251, 1994.
D. Bikel, S. Miller, R. Schwartz, and R. Weischedel. Nymble: A high-performance learning name finder. In The Fifth Conference on Applied Natural Language Processing, pages 194–201. ACM, New York, 1997.
E. Brill. Transformation-based error-driven learning and natural language processing: A case study in part-of-speech tagging. Computational Linguistics, 21(4):543–565, 1995. http://www.cis.upenn.edu/~adwait/penntools.html.
E. Charniak. Statistical techniques for natural language parsing. AI Magazine, 18(4):33–43, 1997.
D. Chiang. Statistical parsing with an automatically-extracted tree adjoining grammar. In Proceedings of the ACL 2000, pages 456–463. ACL, East Stroudsburg, 2000.
J. Earley. An efficient context-free parsing algorithm. Communications of the ACM, 13(2):94–102, 1970.
C. Feldbaum, editor. Wordnet: An Electronic Lexical Database. MIT Press, Cambridge, 1998.
G. Forman. An extensive empirical study of feature selection metrics for text classification. Journal of Machine Learning Research, 3:1289–1305, 2003.
N. Ide and J. Véronis. Word sense disambiguation: The state of the art. Computational Linguistics, 24(1):1–40, 1998.
N. Indurkhya and F. Damerau, editors. Handbook of Natural Language Processing, Second Edition. CRC Press/Taylor and Francis, Boca Raton/London, 2010.
D. Jurafsky and J. Martin. Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition. Pearson, Upper Saddle River, 2008.
T. Kudoh and Y. Matsumoto. Use of support vector learning for chunk identification. In Proceedings of CoNLL-2000 and LLL-2000, pages 142–144. ACL, East Stroudsburg, 2000.
D. Lewis. Feature selection and feature extraction for text categorization. In Proceedings of the Speech and Natural Language Workshop, pages 212–217. Morgan Kaufmann, San Francisco, 1992.
D. Lewis, Y. Yang, T. Rose, and F. Li. RCV1: A new benchmark collection for text categorization research. Journal of Machine Learning Research, 5:361–397, 2004.
M. McCord. Slot grammar: A system for simple construction of practical natural language grammars. In Proceedings of the International Symposium on Natural Language and Logic, pages 118–145. Springer, New York, 1989.
M. Porter. An algorithm for suffix stripping. Program, 14(3):130–137, 1980.
L. Ramshaw and M. Marcus. Text chunking using transformation-based learning. In Proceedings of the Third Workshop on Very Large Corpora, pages 82–94. ACL, East Stroudsburg, 1995.
A. Ratnaparkhi. A maximum entropy part-of-speech tagger. Computational Linguistics, 21(4):543–565, 1995. http://www.cis.upenn.edu/~adwait/penntools.html.
A. Ratnaparkhi. Learning to parse natural language with maximum entropy models. Machine Learning, 34:151–178, 1999.
E. Ray. Learning XML. O’Reilly & Associates, Sebastopol, 2001.
E. Sang and S. Buchholz. Introduction to the CoNLL-2000 shared task: Chunking. In Proceedings of the CoNLL-2000 and LLL-2000, pages 127–132. ACL, East Stroudsburg, 2000.
C.-M. Tan, Y.-F. Wang, and C.-D. Lee. The use of bigrams to enhance text categorization. Information Processing and Management, 38(4):529–546, 2002.
M. Tomita. Efficient Parsing for Natural Language. Kluwer Academic, Dordrecht, 1985.
D. Walker, D. Clements, M. Darwin, and J. Amtrup. Sentence boundary detection: AÂ comparison of paradigms for improving MT quality. In Proceedings of the Eighth Machine Translation Summit. ACL, East Stroudsburg, 2001.
J. Xu and B. Croft. Corpus-based stemming using cooccurrence of word variants. ACM Topics on Information Systems, 16(1):61–81, 1998.
Y. Yang and J. Pedersen. A comparative study of feature selection in text categorization. In Proceedings of the Fourteenth International Conference on Machine Learning, pages 412–420. Morgan Kaufmann, San Francisco, 1997.
T. Zhang, F. Damerau, and D. Johnson. Text chunking based on a generalization of Winnow. Journal of Machine Learning Research, 2(5):615–637, 2002.
T. Zhang, F. Damerau, and D. Johnson. Updating an NLP system to fit new domains: An empirical study on the sentence segmentation problem. In Proceedings of the Seventh Conference on Natural Language Learning, CoNLL-2003, pages 56–62. ACL, East Stroudsburg, 2003.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Copyright information
© 2010 Springer-Verlag London Limited
About this chapter
Cite this chapter
Weiss, S.M., Indurkhya, N., Zhang, T. (2010). From Textual Information to Numerical Vectors. In: Fundamentals of Predictive Text Mining. Texts in Computer Science. Springer, London. https://doi.org/10.1007/978-1-84996-226-1_2
Download citation
DOI: https://doi.org/10.1007/978-1-84996-226-1_2
Publisher Name: Springer, London
Print ISBN: 978-1-84996-225-4
Online ISBN: 978-1-84996-226-1
eBook Packages: Computer ScienceComputer Science (R0)