File(s) under permanent embargo

Beyond tf-idf and cosine distance in documents dissimilarity measure

conference contribution
posted on 2015-01-01, 00:00 authored by Sunil AryalSunil Aryal, Kai Ming Ting, Gholamreza Haffari, Takashi Washio
In vector space model, different types of term weighting schemes are used to adjust bag-of-words document vectors in order to improve the performance of the most widely used cosine distance. Even though the cosine distance with some term weighting schemes result in more reliable (dis)similarity measure in some data sets, it may not perform well in others because of the underlying assumptions of the term weighting schemes. In this paper, we argue that the explicit adjustment of bag-of-words document vectors using term weighting is not required if a data-dependent dissimilarity measure called $$m_p$$-dissimilarity is used. Our empirical result in document retrieval task reveals that $$m_p$$with the simplest binary bag-of-words representation is either better or competitive to the cosine distance with the best performing state-of-the-art term weighting scheme in four widely used benchmark document collections.

History

Event

Information Retrieval Technology. Conference (11th : 2015 : Brisbane, Queensland)

Volume

9460

Series

Lecture Notes in Computer Science

Pagination

400 - 406

Publisher

Springer

Location

Brisbane, Queensland

Place of publication

Cham, Switzerland

Start date

2015-12-02

End date

2015-12-04

ISBN-13

9783319289403

Language

eng

Publication classification

E1.1 Full written paper - refereed

Copyright notice

2015, Springer International Publishing Switzerland

Editor/Contributor(s)

Guido Zuccon, Shlomo Geva, Hideo Joho, Falk Scholer, Aixin Sun, Peng Zhang

Title of proceedings

AIRS 2015 : Information Retrieval Technology: Proceedings of the 11th Asia Information Retrieval Societies Conference