File(s) under permanent embargo
A Novel Approach to Train Diverse Types of Language Models for Health Mention Classification of Tweets
chapter
posted on 2023-03-01, 05:26 authored by PI Khan, Imran RazzakImran Razzak, A Dengel, S AhmedHealth mention classification deals with the disease detection in a given text containing disease words. However, non-health and figurative use of disease words adds challenges to the task. Recently, adversarial training acting as a means of regularization has gained popularity in many NLP tasks. In this paper, we propose a novel approach to train language models for health mention classification of tweets that involves adversarial training. We generate adversarial examples by adding perturbation to the representations of transformer models for tweet examples at various levels using Gaussian noise. Further, we employ contrastive loss as an additional objective function. We evaluate the proposed method on the PHM2017 dataset extended version. Results show that our proposed approach improves the performance of classifier significantly over the baseline methods. Moreover, our analysis shows that adding noise at earlier layers improves models’ performance whereas adding noise at intermediate layers deteriorates models’ performance. Finally, adding noise towards the final layers performs better than the middle layers noise addition.
History
Volume
13530 LNCSPagination
136-147Publisher DOI
ISSN
0302-9743eISSN
1611-3349ISBN-13
9783031159305Language
EnglishEditor/Contributor(s)
Aydin MPublisher
SPRINGER INTERNATIONAL PUBLISHING AGTitle of book
Lecture Notes in Computer ScienceSeries
Lecture Notes in Computer ScienceUsage metrics
Categories
No categories selectedKeywords
Licence
Exports
RefWorks
BibTeX
Ref. manager
Endnote
DataCite
NLM
DC