WEBDIFF: Automated identification of cross-browser issues in web applications

Abstract

Cross-browser (and cross-platform) issues are prevalent in modern web based applications and range from minor cosmetic bugs to critical functional failures. In spite of the relevance of these issues, cross-browser testing of web applications is still a fairly immature field. Existing tools and techniques require a considerable manual effort to identify such issues and provide limited support to developers for fixing the underlying cause of the issues. To address these limitations, we propose a technique for automatically detecting cross-browser issues and assisting their diagnosis. Our approach is dynamic and is based on differential testing. It compares the behavior of a web application in different web browsers, identifies differences in behavior as potential issues, and reports them to the developers. Given a page to be analyzed, the comparison is performed by combining a structural analysis of the information in the page's DOM and a visual analysis of the page's appearance, obtained through screen captures. To evaluate the usefulness of our approach, we implemented our technique in a tool, called WEBDIFF, and used WEBDIFF to identify cross-browser issues in nine real web applications. The results of our evaluation are promising, in that WEBDIFF was able to automatically identify 121 issues in the applications, while generating only 21 false positives. Moreover, many of these false positives are due to limitations in the current implementation of WEBDIFF and could be eliminated with suitable engineering.