Introduction and methodology

Abstract

©2009 EDUCAUSE. Reproduction by permission only. In the second decade of the Internet era, the world is awash in a stunning quantity and variety of information. Exactly how much digital information we’re producing and living alongside is a tough question to answer, but there’s no doubt that the answer involves the kinds of numbers we used to associate mainly with extragalactic distances. A study by the University of California, Berkeley, estimated that in 2002 a total of 5 exabytes (5 billion gigabytes) of new data was recorded on media (including paper)—37,000 times the size of the Library of Congress—and another 18 exabytes flowed through electronic channels.1 The IT research firm IDC, estimating the size of what the company calls the digital universe, produced an even more eye-popping figure: 281 exabytes of digital information created, captured, and replicated in 2007 alone, a quantity IDC expects to grow by 60% annually through 2011.2 These numbers reflect a particularly fast-growing component of image and video capture, but there seems to be hardly any endeavor that isn’t confronted with an exploding body of digital material. Scientific data captured in experiments or generated in computational research has reached heroic proportions, with no end in sight; at the upper end of data density, the Large Hadron Collider at CERN is expected to produce 15 million gigabytes of raw experimental data per year, a DVD every five seconds.3 The more bookish and text-bound side of the academy is undergoing a digital revolution of its own, as Google and other entities produce a searchable online corpus of books that could only have been found at the finest research libraries a decade ago—if there. Perhaps the most remarkable efflorescence of digital content has come from a combination of widening Internet access and web interactivity, a phenomenon almost impossible to quantify but suggested in outline by one estimate that almost 1.7 billion people worldwide now have access to the Internet, and another estimate that the number of available websites exceeds 225 million.4 And that doesn’t count more than 400,000 terabytes of new information generated annually by e-mail!5 Nor are more traditional sorts of structured business data lying still. In a 2007 study by the IT advisory firm Ventana Research, more than half of organizations surveyed reported that their structured business data was growing by 20% or more annually, and their unstructured content was growing faster still.6 For that matter, such neat categories as structured and unstructured data are getting intertwined, as back-end databases dynamically feed websites and XML-related technologies contribute to 2 Introduction and Methodology