Web text mining using a hybrid system

Abstract

This paper presents the research of artificial intelligence techniques based on knowledge discovery in databases (KDD), knowledge discovery in texts, expert systems and artificial neural networks (ANN) applied for evaluation and selection of textual documents found on the World Wide Web. These techniques are useful because nowadays we have a explosive growth of the Web that provides a great amount of documents of many different subjects and the user needs to select these documents regarding to theirs particular interests. We considered the Web as a large data warehouse and applied the KDD fundament and text mining procedures to develop these techniques. The techniques developed are language syntax independent because they do not use the NLP parser and provide an automatic text evaluation based on user profile interests acquired by examples using ANN. Finally, we developed a system using these techniques and compared with a similar commercial system available in the Web.