Openly accessible

Vector representation based on a supervised codebook for Nepali documents classification

Sitaula, Chiranjibi, Basnet, Anish and Aryal, Sunil 2021, Vector representation based on a supervised codebook for Nepali documents classification, PeerJ Computer Science, vol. 7, pp. 1-18, doi: 10.7717/peerj-cs.412.

Attached Files
Name Description MIMEType Size Downloads

Title Vector representation based on a supervised codebook for Nepali documents classification
Author(s) Sitaula, ChiranjibiORCID iD for Sitaula, Chiranjibi orcid.org/0000-0002-4564-2985
Basnet, Anish
Aryal, SunilORCID iD for Aryal, Sunil orcid.org/0000-0002-6639-6824
Journal name PeerJ Computer Science
Volume number 7
Article ID e412
Start page 1
End page 18
Total pages 18
Publisher PeerJ Inc.
Place of publication San Diego, Calif.
Publication date 2021-03-03
ISSN 2376-5992
2376-5992
Keyword(s) Science & Technology
Technology
Computer Science, Artificial Intelligence
Computer Science, Information Systems
Computer Science, Theory & Methods
Computer Science
Text classification
Machine learning
Codebook
Nepali documents
Classification
Feature extraction
Summary Document representation with outlier tokens exacerbates the classification performance due to the uncertain orientation of such tokens. Most existing document representation methods in different languages including Nepali mostly ignore the strategies to filter them out from documents before learning their representations. In this article, we propose a novel document representation method based on a supervised codebook to represent the Nepali documents, where our codebook contains only semantic tokens without outliers. Our codebook is domain-specific as it is based on tokens in a given corpus that have higher similarities with the class labels in the corpus. Our method adopts a simple yet prominent representation method for each word, called probability-based word embedding. To show the efficacy of our method, we evaluate its performance in the document classification task using Support Vector Machine and validate against widely used document representation methods such as Bag of Words, Latent Dirichlet allocation, Long Short-Term Memory, Word2Vec, Bidirectional Encoder Representations from Transformers and so on, using four Nepali text datasets (we denote them shortly as A1, A2, A3 and A4). The experimental results show that our method produces state-of-the-art classification performance (77.46% accuracy on A1, 67.53% accuracy on A2, 80.54% accuracy on A3 and 89.58% accuracy on A4) compared to the widely used existing document representation methods. It yields the best classification accuracy on three datasets (A1, A2 and A3) and a comparable accuracy on the fourth dataset (A4). Furthermore, we introduce the largest Nepali document dataset (A4), called NepaliLinguistic dataset, to the linguistic community.
Language eng
DOI 10.7717/peerj-cs.412
Indigenous content off
Field of Research 0806 Information Systems
HERDC Research category C1 Refereed article in a scholarly journal
Free to Read? Yes
Persistent URL http://hdl.handle.net/10536/DRO/DU:30149544

Connect to link resolver
 
Unless expressly stated otherwise, the copyright for items in DRO is owned by the author, with all rights reserved.

Every reasonable effort has been made to ensure that permission has been obtained for items included in DRO. If you believe that your rights have been infringed by this repository, please contact drosupport@deakin.edu.au.

Versions
Version Filter Type
Citation counts: TR Web of Science Citation Count  Cited 1 times in TR Web of Science
Scopus Citation Count Cited 4 times in Scopus
Google Scholar Search Google Scholar
Access Statistics: 72 Abstract Views, 1 File Downloads  -  Detailed Statistics
Created: Sat, 27 Mar 2021, 11:29:01 EST

Every reasonable effort has been made to ensure that permission has been obtained for items included in DRO. If you believe that your rights have been infringed by this repository, please contact drosupport@deakin.edu.au.