Modeling Search Engine Performance Measurement

The information retrieval (IR) community is rightly proud of its passion for evaluation. This conference has been a welcome refuge when passion becomes obsession. ICTIR's transformation from a largely mathematically based theoretical forum to one that seeks generalizable observations from all areas perfectly suits the needs of IR. However, how much have researchers sought to generalize or model search from evaluation? I will present a set of research papers by others as well as my collaborators and I that since the early 1990s have reported generalizing observations from large scale tests. It's only relatively recently that I've come to realise that these results have been missed by many in the community, yet the models produced carry a great deal of valuable generalizing information about our retrieval systems.