For scientific data discovery: why can't the archive be more like the Web?

The paper addresses the problem of acquiring from scientific data, metadata that is descriptive of the actual content of the data. Scientists can use this content based metadata in subsequent archive searches to find data sets of interest. Such metadata would be especially useful in large scientific archives such as NASA's Earth Observing System Data and Information System (EOSDIS). The paper presents two generic approaches for content based metadata acquisition: target dependent and target independent. Both of these approaches are oriented toward characterizing datasets in terms of the scientific phenomena, such as mesoscale convective systems (severe storms) that they contain. In the target dependent approach, the archived data is mined for particular phenomena of interest and polygons representing the phenomena are stored in a spatial database where they can be used in the data search process. In the target independent approach, data is initially mined for deviations from normal and for trends. This data can then be used for subsequent searches for particular transient phenomena using the deviation data, or for phenomena related to trends. The paper describes results from implementing both of these approaches.