Extracting Named Entities at Web Scale for Competitive Intelligence

Businesses of all sizes have now realized that the Web is an invaluable resource for competitive intelligence, and consequently business decision making. But many have trouble collecting targeted & useful information, and are often further overwhelmed by the time required for analysis & monitoring. On another hand, text mining techniques have become widely used for information analysis in the scientific community in general, and are now ubiquitous in most Web Intelligence fields. With the availability of services such as Google Prediction API, or mature open source software such as GATE, RapidMiner or NLTK, one can expect a much wider adoption of text mining and associated machine learning techniques by expert developers. But how can these techniques benefit to the daily life of a wider business audience? As competitive intelligence is often focused on products, people, customers and competitors, there is an added value for systems providing analytics on these entities, whose recognition is fundamental to text mining and semantic analysis, and consequently is still under active scientific investigation. In this talk we will tour some of the specific requirements and options for building an efficient Web based competitive intelligence system with named entity analytics. We will see how some savvy simplifications can help to overcome common issues such as Web scale and Web content noise, and finally deliver acceptable usability and value for non-specialists, business users.