Cross-Modal Correlation Mining Using Graph Algorithms

View Sample PDF

Author(s): Jia-Yu Pan (Carnegie Mellon University, USA), Hyung-Jeong Yang (Chonnam National University, South Korea)and Christos Faloutsos (Bilkent University, Turkey)
Copyright: 2007
Pages: 25
Source title: Knowledge Discovery and Data Mining: Challenges and Realities
Source Author(s)/Editor(s): Xingquan Zhu (University of Vermont, USA)and Ian Davidson (State University of New York at Albany, USA)
DOI: 10.4018/978-1-59904-252-7.ch004

Keywords: Data Mining and Databases / Information Science Reference / Knowledge Discovery / Library & Information Science

Purchase

View on the publisher's website for pricing and purchasing information.

Abstract

Multimedia objects like video clips or captioned images contain data of various modalities such as image, audio, and transcript text. Correlations across different modalities provide information about the multimedia content, and are useful in applications ranging from summarization to semantic captioning. We propose a graph-based method, MAGIC, which represents multimedia data as a graph and can find cross-modal correlations using “random walks with restarts.” MAGIC has several desirable properties: (a) it is general and domain-independent; (b) it can detect correlations across any two modalities; (c) it is insensitive to parameter settings; (d) it scales up well for large datasets; (e) it enables novel multimedia applications (e.g., group captioning); and (f) it creates opportunity for applying graph algorithms to multimedia problems. When applied to automatic image captioning, MAGIC finds correlations between text and image and achieves a relative improvement of 58% in captioning accuracy as compared to recent machine learning techniques.

The IRMA Community

Research IRM

Cross-Modal Correlation Mining Using Graph Algorithms

Purchase

Abstract

Related Content

IRMA Sponsors