zaidi-ontheeffectivenessof-2020.pdf (8.08 MB)
On the effectiveness of discretizing quantitative attributes in linear classifiers
journal contribution
posted on 2020-10-01, 00:00 authored by Nayyar ZaidiNayyar Zaidi, Yang Du, Geoffrey I WebbLinear models in machine learning are extremely computational efficient but they have high representation bias due to non-linear nature of many real-world datasets. In this article, we show that this representation bias can be greatly reduced by discretization. Discretization is a common procedure in machine learning that is used to convert a quantitative attribute into a qualitative one. It is often motivated by the limitation of some learners to handle qualitative data. Since discretization looses information (as fewer distinctions among instances are possible using discretized data relative to undiscretized data) – where discretization is not essential, it might appear desirable to avoid it, and typically, it is avoided. However, in the past, it has been shown that discretization can leads to superior performance on generative linear models, e.g., naive Bayes. This motivates a systematic study of the effects of discretizing quantitative attributes for discriminative linear models, as well. In this article, we demonstrate that, contrary to prevalent belief, discretization of quantitative attributes, for discriminative linear models, is a beneficial pre-processing step, as it leads to far superior classification performance, especially on bigger datasets, and surprisingly, much better convergence, which leads to better training time. We substantiate our claims with an empirical study on 52 benchmark datasets, using three linear models optimizing different objective functions.
History
Journal
IEEE AccessVolume
8Pagination
198856 - 198871Publisher
Institute of Electrical and Electronics Engineers (IEEE)Location
Piscataway, N. J.Publisher DOI
Link to full text
eISSN
2169-3536Language
engPublication classification
C1 Refereed article in a scholarly journalCopyright notice
2020, The AuthorsUsage metrics
Categories
No categories selectedKeywords
discretizationclassificationlogistic regressionsupport vector classifierartificial neuronbig datasetsbias-variance analysisScience & TechnologyTechnologyComputer Science, Information SystemsEngineering, Electrical & ElectronicTelecommunicationsComputer ScienceEngineeringMISCLASSIFICATIONDICHOTOMIZATION
Licence
Exports
RefWorks
BibTeX
Ref. manager
Endnote
DataCite
NLM
DC