Databases on the web: national web domain survey

Abstract

The deep Web, the part of the Web consisting of web pages filled with information from myriads of online databases, is to date relatively unexplored. Even its basic characteristics such as, for instance, the number of searchable databases on the Web are disputable. In this paper, we address the problem of accurate estimation of the deep Web by sampling one national web domain. We report some of our results obtained when surveying the Russian Web. The survey findings, namely the size estimates of the deep Web, could be useful for further studies to handle data in the deep Web.