A Software Tool for Collecting Data from Online Auctions

Online auctions are increasingly becoming the platform of choice for dubious sellers to engage anonymously in fraudulent behaviour.  While researchers are noble in their efforts to devise mechanisms to counter auction fraud, they are often frustrated by the lack of available auction data.  Such data is an invaluable tool to gain insight about fraudulent traits and for testing proposed security remedies. This is compounded by online auction sources being non-cooperative in providing auction data, usually citing "security and privacy'' as reasons for not wanting to help.  This paper presents a software tool that can extract data from various online auction sources.  The system is able to collect all the data for a given search criteria on auctions that have completed, and also returns later to collect the data from ongoing auctions once they have completed (without user intervention).  We share our experiences from the development process and describe the challenges that must be overcome to successfully set up such a system. The data collected is used to analyse the behaviour and bidding patterns of sellers and buyers that are engaged in online auctions.  The work presented in this paper represents the first serious attempt at creating an openly available software tool and establishing a repository of online auction data that will be free for use by other researchers.