Abstract
Evaluation of ontology alignments is in practice done in two ways: (1) assessing individual correspondences and (2) comparing the alignment to a reference alignment. However, this type of evaluation does not guarantee that an application which uses the alignment will perform well. In this paper, we contribute to the current ontology alignment evaluation practices by proposing two alternative evaluation methods that take into account some characteristics of a usage scenario without doing a full-fledged end-to-end evaluation. We compare different evaluation approaches in three case studies, focussing on methodological issues. Each case study considers an alignment between a different pair of ontologies, ranging from rich and well-structured to small and poorly structured. This enables us to conclude on the use of different evaluation approaches in different settings.
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Ashpole, B., Ehrig, M., Euzenat, J., Stuckenschmidt, H. (eds.): Proceedings of the K-CAP 2005 Workshop on Integrating Ontologies (2005)
Brink, W.P., van den Koele, P.: Statistiek, Boom, Amsterdam, The Netherlands, vol. 3 (2002) ISBN 90 5352 705 2
Budanitsky, A., Hirst, G.: Semantic distance in wordnet: an experimental application oriented evaluation of five measures. In: Proceedings of the NACCL 2001 Workshop on WordNet and other lexical resources, Pittsburgh, PA, pp. 29–34 (2001)
Ehrig, M., Euzenat, J.: Relaxed precision and recall for ontology matching. In: Ashpole, et al. [1]
Euzenat, J.: Semantic precision and recall for ontology alignment evaluation. In: Veloso, M.M. (ed.) Proceedings of the International Joint Conferences on Artificial Intelligence, pp. 348–353 (2007)
Euzenat, J., Shvaiko, P.: Ontology matching. Springer, Heidelberg (2007)
Euzenat, J., Stuckenschmidt, H., Yatskevich, M.: Introduction to the ontology alignment evaluation 2005. In: Ashpole, et al. [1] (2005)
Euzenat, J., Mochol, M., Shvaiko, P., Stuckenschmidt, H., Šváb, O., Svátek, V., van Hage, W.R., Yatskevich, M.: Results of the OAEI 2006. In: Ashpole, B., Ehrig, M., Euzenat, J., Stuckenschmidt, H. (eds.) Ontology Matching. CEUR Workshop Proceedings, vol. 225 (2006)
Euzenat, J., Isaac, A., Meilicke, C., Shvaiko, P., Stuckenschmidt, H., Šváb, O., Svátek, V., van Hage, W.R., Yatskevich, M.: First results of the OAEI 2007. In: Ashpole, B., Ehrig, M., Euzenat, J., Stuckenschmidt, H. (eds.) Ontology Matching. CEUR Workshop Proc. (2007)
Hu, W., Qu, Y.: Discovering simple mappings between relational database schemas and ontologies. In: Proceedings of the International Semantic Web Conference, pp. 225–238 (2007)
Isaac, A., Wang, S.: Evaluation issues at the library testcase of oaei 2007. In: ESWC 2008 (accepted for publication, 2008)
Isaac, A., Zinn, C., Matthezing, H., van der Meij, L., Schlobach, S., Wang, S.: The value of usage scenarios for thesaurus alignment in cultural heritage context. In: Proceedings of International Workshop on Cultural Heritage on the Semantic Web, ISWC 2007, Korea (2007)
Kalfoglou, Y., Schorlemmer, M.: Ontology mapping: the state of the art. The Knowledge Engineering Review 18(1), 1–31 (2003)
Kekäläinen, J., Järvelin, K.: Using graded relevance assessments in ir evaluation. Journal of the American Society for Information Science and Technology 53(13) (2002) ISSN 1532-2882
Leacock, C., Chodorow, M.: Combining Local Context and WordNet Similarity for Word Sense Identification, ch. 11, pp. 265–285. MIT Press, Cambridge (1998)
Peterson, T.: Introduction to the Art and Architecture Thesaurus. Oxford University Press, Oxford (1994)
Schreiber, A.Th., Amin, A., van Assem, M., de Boer, V., Hardman, L., Hildebrand, M., Hollink, L., Huang, Z., van Kersen, J., de Niet, M., Omelayenko, B., van Ossenbruggen, J., Siebes, R., Taekema, J., Wielemaker, J., Wielinga, B.J.: Multimedian e-culture demonstrator. In: The Semantic Web Challenge at the Fifth International Semantic Web Conference, Athens, GA, USA (November 2006)
Shvaiko, P., Euzenat, J.: A survey of schema-based matching approaches. Journal on Data Semantics 3730, 146–171 (2005)
van Assem, M., Gangemi, A., Schreiber, G.: RDF/OWL Representation of WordNet. W3C Working Draft, World Wide Web Consortium (June 2006)
van Hage, W.R., Isaac, A., Aleksovski, Z.: Sample evaluation of ontologymatching systems. In: Proceedings of the Fifth International Evaluation of Ontologies and Ontology-based Tools, Busan, Korea (2007)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Hollink, L., van Assem, M., Wang, S., Isaac, A., Schreiber, G. (2008). Two Variations on Ontology Alignment Evaluation: Methodological Issues. In: Bechhofer, S., Hauswirth, M., Hoffmann, J., Koubarakis, M. (eds) The Semantic Web: Research and Applications. ESWC 2008. Lecture Notes in Computer Science, vol 5021. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-68234-9_30
Download citation
DOI: https://doi.org/10.1007/978-3-540-68234-9_30
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-68233-2
Online ISBN: 978-3-540-68234-9
eBook Packages: Computer ScienceComputer Science (R0)