Eestikeelsete veebitekstide automaatne liigitamine

Abstract

Internet on oluline keeleressurss, mille uheks keeleteaduslikuks ja keeletehnoloogiliseks kasutusvoimaluseks on seal leiduvate tekstide koondamine keelekorpuseks. Kuid taisautomaatselt korjatud korpusega seistakse uudse situatsiooni ees: olemas on palju andmeid, ent pole tapselt teada, millist keelematerjali need sisaldavad. Loomuliku keele uurimise ja tootlemise seisukohalt on vajalik tekstide eristamine tekstiliigiti, sest sellest soltub sobivate tootlusvahendite valik. Artiklis kirjeldame tekstiliikide eristamise lihtsustatud versiooni: korpuse Estonian Web 2013 (etTenTen13) binaarse klassifitseerimise katset, mille eesmark oli liigitada tekstid kirjakeele normi jargivateks ja mittejargivateks. Treeningandmetes kasutasime kirjakeele esindajana Tasakaalus korpust ja kirjakeele normi mitte jargivate tekstide esindajana Uue meedia korpust ning testandmetena kasitsi liigitatud Estonian Web 2013 alamkorpust. Klassifitseerimismudelite loomisel rakendasime erinevaid juhendatud masinoppe algoritme ning tunnustena sone- hulkasid. Klassifitseerimismudelite kvaliteeti hindasime 10-kordse ristvalideerimise teel, kus parima tulemuse andis tehisnarvivorkudel pohinev algoritm, mis 99% tapsusega liigitas dokumendi oigesse klassi. Seejarel katsetasime mudeleid kasitsi liigitatud Estonian Web 2013 testkorpusel, kus parima tulemuse andis taas tehisnarvivorkudel pohinev algoritm tapsusega 74%. *** Classifying Estonian Web texts Due to the size of the Internet and the multitude of traditional and new genres there has been an increasing interest in automatic genre classification. Labelling texts in natural language processing is essential because this allows us to select more appropriate language models for the analysis. The aim of the article is to describe and present the results of automatically classifying Estonian Web 2013 texts. We evalued the quality of different classification models on our training and manually labelled test set. Most of the research on automatic classification has focused on classifying multiple genres, while our objective was to do a binary classification. We set out to classify Estonian Web 2013 texts based on whether they are canonical or not. For training we used the Balanced Corpus to represent canonical language and the New Media Corpus to represent non-canonical language. Due to the non-availability of a binary labelled subcorpus of Estonian Web 2013 texts, we compiled it ourselves by manually labelling it. For classification we used different supervised machine learning algorithms and for features a simple Bag of Words method. The results obtained from the preliminary experiments show that neural networks outperformed other machine learning algorithms achieving over 0.7 on accuracy. The overall results of this study indicate that in order to increase the accuracy of the classifiers, new features should be added (e.g POS count, sentences per paragraph, words per sentence, uppercase and lowercase letters per sentence etc.). Our best model, the neural network classifier, achieved an accuracy of 0.99 on a training set but only a little over 0.74 on the test set. This suggests that future work requiers a bigger and more appropriate training set. The manually labelling task showed us that the transition from canonical to non-canonical is very smooth. Current models produce a score between 0 and 1, defining if the item belongs to a class or not. Therefore, the classification models must be programmed to be more predictive so that the predictions can be tuned by selecting a threshold.