A graph theory-based online keywords model for image semantic extraction

Jing Wang, Zhijie Xu

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Image captions and keywords are the semantic descriptions of the dominant visual content features in a targeted visual scene. Traditional image keywords extraction processes involves intensive data- and knowledge-level operations by using computer vision and machine learning techniques. However, recent studies have shown that the gap between pixel-level processing and the semantic definition of an image is difficult to bridge by counting only the visual features. In this paper, augmented image semantic information has been introduced through harnessing functions of online image search engines. A graphical model named as the "Head-words Relationship Network" (HWRN) has been devised for tackling the aforementioned problems. The proposed algorithm starts from retrieving online images of similarly visual features from the input image, the text content of their hosting webpages are then extracted, classified and analysed for semantic clues. The relationships of those "head-words" from relevant webpages can then be modelled and quantified using linguistic tools. Experiments on the prototype system have proven the effectiveness of this novel approach. Performance evaluation over benchmarking state-of-theart approaches has also shown satisfactory results and promising future applications.

LanguageEnglish
Title of host publication2016 Symposium on Applied Computing, SAC 2016
PublisherAssociation for Computing Machinery (ACM)
Pages67-72
Number of pages6
ISBN (Electronic)9781450337397
DOIs
Publication statusPublished - 2016
Event31st Annual Association for Computing Machinery Symposium on Applied Computing - Pisa, Italy
Duration: 4 Apr 20168 Apr 2016
Conference number: 31
https://www.sigapp.org/sac/sac2016/ (Link to Symposium Website )

Conference

Conference31st Annual Association for Computing Machinery Symposium on Applied Computing
Abbreviated titleSAC 2016
CountryItaly
CityPisa
Period4/04/168/04/16
Internet address

Fingerprint

Graph theory
Semantics
Benchmarking
Search engines
Linguistics
Computer vision
Learning systems
Pixels
Processing
Experiments

Cite this

Wang, J., & Xu, Z. (2016). A graph theory-based online keywords model for image semantic extraction. In 2016 Symposium on Applied Computing, SAC 2016 (pp. 67-72). Association for Computing Machinery (ACM). https://doi.org/10.1145/2851613.2851633
Wang, Jing ; Xu, Zhijie. / A graph theory-based online keywords model for image semantic extraction. 2016 Symposium on Applied Computing, SAC 2016. Association for Computing Machinery (ACM), 2016. pp. 67-72
@inproceedings{235fa3232bf84ab782cd2cde493cd4ca,
title = "A graph theory-based online keywords model for image semantic extraction",
abstract = "Image captions and keywords are the semantic descriptions of the dominant visual content features in a targeted visual scene. Traditional image keywords extraction processes involves intensive data- and knowledge-level operations by using computer vision and machine learning techniques. However, recent studies have shown that the gap between pixel-level processing and the semantic definition of an image is difficult to bridge by counting only the visual features. In this paper, augmented image semantic information has been introduced through harnessing functions of online image search engines. A graphical model named as the {"}Head-words Relationship Network{"} (HWRN) has been devised for tackling the aforementioned problems. The proposed algorithm starts from retrieving online images of similarly visual features from the input image, the text content of their hosting webpages are then extracted, classified and analysed for semantic clues. The relationships of those {"}head-words{"} from relevant webpages can then be modelled and quantified using linguistic tools. Experiments on the prototype system have proven the effectiveness of this novel approach. Performance evaluation over benchmarking state-of-theart approaches has also shown satisfactory results and promising future applications.",
keywords = "Graphical model, Image semantic, Linguistic hyponym trees, Online search engine",
author = "Jing Wang and Zhijie Xu",
year = "2016",
doi = "10.1145/2851613.2851633",
language = "English",
pages = "67--72",
booktitle = "2016 Symposium on Applied Computing, SAC 2016",
publisher = "Association for Computing Machinery (ACM)",
address = "United States",

}

Wang, J & Xu, Z 2016, A graph theory-based online keywords model for image semantic extraction. in 2016 Symposium on Applied Computing, SAC 2016. Association for Computing Machinery (ACM), pp. 67-72, 31st Annual Association for Computing Machinery Symposium on Applied Computing, Pisa, Italy, 4/04/16. https://doi.org/10.1145/2851613.2851633

A graph theory-based online keywords model for image semantic extraction. / Wang, Jing; Xu, Zhijie.

2016 Symposium on Applied Computing, SAC 2016. Association for Computing Machinery (ACM), 2016. p. 67-72.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - A graph theory-based online keywords model for image semantic extraction

AU - Wang, Jing

AU - Xu, Zhijie

PY - 2016

Y1 - 2016

N2 - Image captions and keywords are the semantic descriptions of the dominant visual content features in a targeted visual scene. Traditional image keywords extraction processes involves intensive data- and knowledge-level operations by using computer vision and machine learning techniques. However, recent studies have shown that the gap between pixel-level processing and the semantic definition of an image is difficult to bridge by counting only the visual features. In this paper, augmented image semantic information has been introduced through harnessing functions of online image search engines. A graphical model named as the "Head-words Relationship Network" (HWRN) has been devised for tackling the aforementioned problems. The proposed algorithm starts from retrieving online images of similarly visual features from the input image, the text content of their hosting webpages are then extracted, classified and analysed for semantic clues. The relationships of those "head-words" from relevant webpages can then be modelled and quantified using linguistic tools. Experiments on the prototype system have proven the effectiveness of this novel approach. Performance evaluation over benchmarking state-of-theart approaches has also shown satisfactory results and promising future applications.

AB - Image captions and keywords are the semantic descriptions of the dominant visual content features in a targeted visual scene. Traditional image keywords extraction processes involves intensive data- and knowledge-level operations by using computer vision and machine learning techniques. However, recent studies have shown that the gap between pixel-level processing and the semantic definition of an image is difficult to bridge by counting only the visual features. In this paper, augmented image semantic information has been introduced through harnessing functions of online image search engines. A graphical model named as the "Head-words Relationship Network" (HWRN) has been devised for tackling the aforementioned problems. The proposed algorithm starts from retrieving online images of similarly visual features from the input image, the text content of their hosting webpages are then extracted, classified and analysed for semantic clues. The relationships of those "head-words" from relevant webpages can then be modelled and quantified using linguistic tools. Experiments on the prototype system have proven the effectiveness of this novel approach. Performance evaluation over benchmarking state-of-theart approaches has also shown satisfactory results and promising future applications.

KW - Graphical model

KW - Image semantic

KW - Linguistic hyponym trees

KW - Online search engine

UR - http://www.scopus.com/inward/record.url?scp=84975796681&partnerID=8YFLogxK

U2 - 10.1145/2851613.2851633

DO - 10.1145/2851613.2851633

M3 - Conference contribution

SP - 67

EP - 72

BT - 2016 Symposium on Applied Computing, SAC 2016

PB - Association for Computing Machinery (ACM)

ER -

Wang J, Xu Z. A graph theory-based online keywords model for image semantic extraction. In 2016 Symposium on Applied Computing, SAC 2016. Association for Computing Machinery (ACM). 2016. p. 67-72 https://doi.org/10.1145/2851613.2851633