A private living lab for requirements based evaluation

A "Living Lab" is described as an open innovation space for the cooperation of users, researchers and even companies to participate in a common process to develop innovative solutions. An architecture for a living lab for IR has been proposed in [1]. In this paper we propose a method and system that foregoes the inherent openness of the living lab and implements a private living lab to enable the cooperation between a research department, agile software development, and requirements engineering/quality assurance. This allows the research department to overcome the limitations of the usual approach to gold standard-based evaluation, while preserving its positive aspects. The definition of a private living lab may be seen along the lines of the separation of public and private clouds in cloud computing.