CWrap: web wrapping using context variables

Abstract

A procedure that extracts data from a data source is called wrapper. In some applications identifying the desired data is better served using a wrapping language rather than an unsupervised method. In this paper, we propose a novel wrapping language, called CWrap. In this language, various types of features (syntactical, semantic, visual and densitometric) can be employed in the extraction rules to identify the items of interest. Moreover, the context in which the desired items appear is specified using variables called context variables. Context variables enable the user to express different types of contextual dependencies (structural, visual and semantical) in a consistent way. They are set under certain conditions by one rule and are used later to form the contextual conditions for another extraction rule. This allows the user to organise the extraction rules in a hierarchical structure, from general to more specific rules. We also present a visual development toolkit which enables the user to develop and debug a wrapper visually and assembling it in an incremental manner.