Evaluating Semantic Metrics on Tasks of Concept Similarity

View Sample PDF

Author(s): Hansen A. Schwartz (University of Pennsylvania, USA)and Fernando Gomez (University of Central Florida, USA)
Copyright: 2012
Pages: 17
Source title: Cross-Disciplinary Advances in Applied Natural Language Processing: Issues and Approaches
Source Author(s)/Editor(s): Chutima Boonthum-Denecke (Hampton University, USA), Philip M. McCarthy (The University of Memphis, USA)and Travis Lamkin (University of Memphis, USA)
DOI: 10.4018/978-1-61350-447-5.ch021

Keywords: Artificial Intelligence / Computational Linguistics / Computer Science & IT / Information Science Reference

Purchase

View Evaluating Semantic Metrics on Tasks of Concept Similarity on the publisher's website for pricing and purchasing information.

Abstract

In this study, first, concept similarity measures are evaluated over human judgments by using existing sets of word similarity pairs that we annotated with word senses. Next, an application-oriented study is presented to evaluate semantic metrics based on integration into an algorithm, first focused on the task of concept similarity then on the task of concept relatedness. The results found no single measure to be most significantly correlated with human-judgments, while an information content-based measure clearly lead to the best results in the application-oriented task of concept similarity. Reinforcing the difference between tasks of concept similarity and concept relatedness, the best measure for an application-oriented task of concept relatedness was a gloss-based relatedness measure rather than a similarity measure. A major conclusion of this work is that similarity measures may perform differently if embedded in specific applications than if they are compared with human judgments.

The IRMA Community

Research IRM

Evaluating Semantic Metrics on Tasks of Concept Similarity

Purchase

Abstract

Related Content

IRMA Sponsors