Automatic Keyword Annotation System Using    Newspapers

Tomoki Takada; Mizuki Arai; Tomohiro Takagi

doi:10.20965/jaciii.2014.p0340

single-jc.php

« previous

JACIII Vol.18 No.3 pp. 340-346

doi: 10.20965/jaciii.2014.p0340

(2014)

Paper:

Views over last 60 days: 914

Automatic Keyword Annotation System Using Newspapers

Tomoki Takada, Mizuki Arai, and Tomohiro Takagi

Department of Computer Science, Meiji University, 1-1-1 Higashimita, Tama-ku, Kawasaki-shi, Kanagawa 214-8571, Japan

Received:

October 13, 2013

Accepted:

January 31, 2014

Published:

May 20, 2014

Keywords:

keyword annotation, confabulation theory, the nikkei

Abstract

Nowadays, an increasingly large amount of information exists on the web. Therefore, a method is needed that enables us to find necessary information quickly because this is becoming increasingly difficult for users. To solve this problem, information retrieval systems like Google and recommendation systems like that on Amazon are used. In this paper, we focus on information retrieval systems. These retrieval systems require index terms, which affect the precision of retrieval. Two methods generally decide index terms. One is analyzing a text using natural language processing and deciding index terms using varying amounts of statistics. The other is someone choosing document keywords as index terms. However, the latter method requires too much time and effort and becomes more impractical as information grows. Therefore, we propose the Nikkei annotator system, which is based on the model of the human brain and learns patterns of past keyword annotation and automatically outputs keywords that users prefer. The purposes of the proposed method are automating manual keyword annotation and achieving high speed and high accuracy keyword annotation. Experimental results showed that the proposed method is more accurate than TFIDF and Naive Bayes in P@5 and P@10. Moreover, these results also showed that the proposed method could annotate about 19 times faster than Naive Bayes.

Cite this article as:

T. Takada, M. Arai, and T. Takagi, “Automatic Keyword Annotation System Using Newspapers,” J. Adv. Comput. Intell. Intell. Inform., Vol.18 No.3, pp. 340-346, 2014.

Data files:

References

[1] M. Hamaguchi, “YOMIDAS REKISHIKAN, new database service of japanese newspaper and YOMIDAS YOGO JISHO, thesaurus by The Yomiuri Shimbun,” J. of Information Processing and Management, Vol.52, No.3, June 2009.
[2] M. Ishii, “Auto-indexing system in Nihon Keizai Shinbun, Inc.,” Information Science and Technology Association, Vol.42, No.11, pp. 1058-1064, November 1992.
[3] Q. Zadeh et al., “Semi-Supervised Technical Term Tagging With Minimal User Feedback,” LREC, pp. 617-621, 2012.
[4] H. Nakagawa et al. “Automatic Term Recognition based on Statistics of Compound Nouns and their Components,” Terminology, Vol.9, No.2, pp. 201-219, 2001.
[5] M. Utiyama et al. “Using Author Keywords for Automatic Term Recognition,” Terminology, Vol.6, No.2, pp. 313-326, 2000.
[6] A. Schenker et al., “Classification of Web Documents Using a Graph Model,” Proc. of the 7th Int. Conf. on Document Analysis and Recognition, Scotland, Computer Society Press, 2003.
[7] S. Bleik et al., “CGM: A Biomedical Text Categorization Approach Using Concept Graph Mining,” Proc. IEEE Int. Conf. on Bioinformatics and Biomedicine Workshop, pp. 38-43, 2009.
[8] M. E. Maron, “Automatic Indexing: An Experimental Inquiry,” J. ACM, Vol.8, No.3, pp. 404-417, 1961.
[9] T. Joachims, “Text Categorization with Support Vector Machines: Learning with Many Relevant Features,” Proc. European Conf. on Machine Learning, pp. 137-142, 1998.
[10] B. V. Dasarathy, “Nearrest Neighbor (NN) Norms: NN Pattern Classification Techniques,” IEEE Press, 1991.
[11] C. Apte et al., “Automated Learning of Decision Rules for Text Categorization,” ACM Trans. Information Systems, Vol.12, No.3, pp. 233-251, 1994.
[12] S. Bleik et al., “Text Categorization of Biomedical Data Sets Using Graph Kernels and a Controlled Vocaburary,” IEEE/ACM Trans. on Computational Biology and Bioinformatics, 2013.
[13] D. Zhang andW. S. Lee, “Extracting Key-Substring-Group Features for Text Classification,” Proc. of the 12th ACM SIGKDD Int. Conf. on Knowledge Discovery and Data Mining, pp. 474-483, 2006.
[14] R. Hecht-Nielsen, “Confabulation Theory,” Springer-Verlag: Heidelberg, 2007.
[15] R. Hecht-Nielsen, “Cogent confabulation,” Neural Networks, Vol.18, pp. 111-115, 2005.
[16] R. Hecht-Nielsen, “Confabulation theory,” UCSD Institute for Neural Computation Technical Report #0501, 2005.
[17] http://nikkei.com [Accessed September 1, 2013].

This article is published under a Creative Commons Attribution-NoDerivatives 4.0 Internationa License.

[1] [1] M. Hamaguchi, “YOMIDAS REKISHIKAN, new database service of japanese newspaper and YOMIDAS YOGO JISHO, thesaurus by The Yomiuri Shimbun,” J. of Information Processing and Management, Vol.52, No.3, June 2009.

[2] [2] M. Ishii, “Auto-indexing system in Nihon Keizai Shinbun, Inc.,” Information Science and Technology Association, Vol.42, No.11, pp. 1058-1064, November 1992.

[3] [3] Q. Zadeh et al., “Semi-Supervised Technical Term Tagging With Minimal User Feedback,” LREC, pp. 617-621, 2012.

[4] [4] H. Nakagawa et al. “Automatic Term Recognition based on Statistics of Compound Nouns and their Components,” Terminology, Vol.9, No.2, pp. 201-219, 2001.

[5] [5] M. Utiyama et al. “Using Author Keywords for Automatic Term Recognition,” Terminology, Vol.6, No.2, pp. 313-326, 2000.

[6] [6] A. Schenker et al., “Classification of Web Documents Using a Graph Model,” Proc. of the 7th Int. Conf. on Document Analysis and Recognition, Scotland, Computer Society Press, 2003.

[7] [7] S. Bleik et al., “CGM: A Biomedical Text Categorization Approach Using Concept Graph Mining,” Proc. IEEE Int. Conf. on Bioinformatics and Biomedicine Workshop, pp. 38-43, 2009.

[8] [8] M. E. Maron, “Automatic Indexing: An Experimental Inquiry,” J. ACM, Vol.8, No.3, pp. 404-417, 1961.

[9] [9] T. Joachims, “Text Categorization with Support Vector Machines: Learning with Many Relevant Features,” Proc. European Conf. on Machine Learning, pp. 137-142, 1998.

[10] [10] B. V. Dasarathy, “Nearrest Neighbor (NN) Norms: NN Pattern Classification Techniques,” IEEE Press, 1991.

[11] [11] C. Apte et al., “Automated Learning of Decision Rules for Text Categorization,” ACM Trans. Information Systems, Vol.12, No.3, pp. 233-251, 1994.

[12] [12] S. Bleik et al., “Text Categorization of Biomedical Data Sets Using Graph Kernels and a Controlled Vocaburary,” IEEE/ACM Trans. on Computational Biology and Bioinformatics, 2013.

[13] [13] D. Zhang andW. S. Lee, “Extracting Key-Substring-Group Features for Text Classification,” Proc. of the 12th ACM SIGKDD Int. Conf. on Knowledge Discovery and Data Mining, pp. 474-483, 2006.

[14] [14] R. Hecht-Nielsen, “Confabulation Theory,” Springer-Verlag: Heidelberg, 2007.

[15] [15] R. Hecht-Nielsen, “Cogent confabulation,” Neural Networks, Vol.18, pp. 111-115, 2005.

[16] [16] R. Hecht-Nielsen, “Confabulation theory,” UCSD Institute for Neural Computation Technical Report #0501, 2005.

[17] [17] http://nikkei.com [Accessed September 1, 2013].