AJAX Crawling Scheme Based on Document Object Model

Abstract

Many web applications adopt AJAX to enhance their user experience. But AJAX has a number of properties making it extremely difficult for traditional search engines to crawl. Google's AJAX crawling scheme suggests webmasters change their website architectures and add additional codes. It is so complicated that it be currently supported only by Google. The paper proposed an AJAX crawling scheme based on DOM and breadth-first AJAX crawling algorithm. It can build a State Transition Graph (STG) of an AJAX web application by tracking the changes of DOM tree triggered by some AJAX events, and then generate a static mirror site of the original AJAX web application.