Yirey Suh, Jae-Myung Yu, Jonghoon Mo, Cheongtag Kim
{"title":"A Comparison of Oversampling Methods on Imbalanced Topic Classification of Korean News Articles","authors":"Yirey Suh, Jae-Myung Yu, Jonghoon Mo, Cheongtag Kim","doi":"10.17791/JCS.2017.18.4.391","DOIUrl":null,"url":null,"abstract":"Machine learning has progressed to match human performance, including the field of text classification. However, when training data are imbalanced, classifiers do not perform well. Oversampling is one way to overcome the problem of imbalanced data and there are many oversampling methods that can be conveniently implemented. While comparative researches of oversampling methods on non-text data have been conducted, studies comparing oversampling methods under a unifying framework on text data are scarce. This study finds that while oversampling methods generally improve the performance of classifiers, similarity is an important factor that influences the performance of classifiers on imbalanced and resampled data.","PeriodicalId":135438,"journal":{"name":"The Journal of Cognitive Science","volume":null,"pages":null},"PeriodicalIF":0.0000,"publicationDate":"2017-12-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"24","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"The Journal of Cognitive Science","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.17791/JCS.2017.18.4.391","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 24
Abstract
Machine learning has progressed to match human performance, including the field of text classification. However, when training data are imbalanced, classifiers do not perform well. Oversampling is one way to overcome the problem of imbalanced data and there are many oversampling methods that can be conveniently implemented. While comparative researches of oversampling methods on non-text data have been conducted, studies comparing oversampling methods under a unifying framework on text data are scarce. This study finds that while oversampling methods generally improve the performance of classifiers, similarity is an important factor that influences the performance of classifiers on imbalanced and resampled data.