Detecting Web Content Function Using Generalized Hidden Markov Model

Abstract

Web content function indicates authors' intension towards the purpose of the content and therefore plays an important role for Web information processing. In this paper we propose a generalized hidden Markov model which extends traditional hidden Markov model for Web content function detection. By incorporating multiple emission features and detecting state transition sequence based on layout structure, generalized hidden Markov model can effectively make use of Web-specific information and achieve better performance comparing to traditional hidden Markov model. Comparing to previous approaches on function detection, our approach has the advantages of domain-independency and extensibility for other applications. Experiments show promising results with our approach