Automatic visual theme discovery from joint image and text corpora

Ke Sun, Xianxu Hou, Qian Zhang, Guoping Qiu

Research output: Chapter in Book or Report/Conference proceedingConference Proceedingpeer-review

Abstract

This paper presents an unsupervised visual theme discovery framework as a better (more compact and effective) alternative for semantic representation of visual contents. Firstly, a tag filtering algorithm was proposed focusing on the tag's ability of visual content description. Then a spectral clustering algorithm is applied to cluster tags into visual themes based on their visual similarity and semantic similarity measures. User studies have been conducted to evaluate the effectiveness and rationality of the discovered visual themes and obtain promising results. Additionally, two common computer vision tasks, example based image search and keyword based image search to explore potential applications of the proposed framework. The experimental results show that visual themes significantly outperform tags on semantic image understanding and achieve state-of-art performance in these two tasks.

Original languageEnglish
Title of host publicationProceedings - 2017 2nd International Conference on Multimedia and Image Processing, ICMIP 2017
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages220-224
Number of pages5
ISBN (Electronic)9781509059546
DOIs
Publication statusPublished - 15 Dec 2017
Externally publishedYes
Event2nd International Conference on Multimedia and Image Processing, ICMIP 2017 - Wuhan, Hubei, China
Duration: 17 Mar 201719 Mar 2017

Publication series

NameProceedings - 2017 2nd International Conference on Multimedia and Image Processing, ICMIP 2017
Volume2017-January

Conference

Conference2nd International Conference on Multimedia and Image Processing, ICMIP 2017
Country/TerritoryChina
CityWuhan, Hubei
Period17/03/1719/03/17

Keywords

  • Convolution neural network
  • Image retrieval
  • Random forest
  • Visual theme discovery
  • Word embedding

Fingerprint

Dive into the research topics of 'Automatic visual theme discovery from joint image and text corpora'. Together they form a unique fingerprint.

Cite this