Using online user behavior to predict demographics

Videology, an online video advertising company, is often unable to obtain gender information about incoming online advertisement requests. They purchase aggregate gender statistics on groups of requests from a third party. This project explores creating groups of requests in which at least 80% of the advertisement requests have the same gender using: 1) traditional clustering algorithms, 2) iterative linear regression algorithm - ITRA - and 3) qualitative clustering algorithm (ROCK). In all cases, the data used was either browsing history data or synthetic attributes created by dimensionality reduction to more simply describe that history. These three approaches were unable to consistently create the desired gender discrimination. None of these three approaches proved to be the preferred alternative as the performance of each varied drastically as the test data set, and even subsets of that test data set changed. However, amongst the data sets used, these methods were able in some instances to create small buckets (less than 3,000 requests) with the desired gender distribution. The success or failure of these algorithms was dependent upon how similar individual requests were to one another (i.e. how many attributes were on average shared between requests). The approaches performed better in those instances in which more attributes were shared between requests, i.e., the requests contained information that allowed for the classification of the requests.