WISE: a visual tool for automatic extraction of objects from World Wide Web

Abstract

Automatic identification and extraction of interesting objects instead of just related Web pages is useful to users of the World Wide Web. For example, an instructor teaching a data mining course may be interested in finding all text books, lecture notes, slides, homeworks and term projects from Web pages of all data mining and their related courses. In this paper, we present a tool WISE for extraction of objects from Web pages, which are of interest. In WISE, objects in HTML pages are defined as nodes in an extension of DOM trees. Extraction of objects is then formulated as the problem of classification of tree nodes. We develop a mechanism for object specifications, and a specific-to-general (bottom-up) search algorithm for learning object specifications. Experiment shows that WISE works well for extracting objects such as, for example, extracting items in paper lists, and extracting lists of lecture notes in algorithms courses.