Designing evaluation beyond evaluating design: measuring success in cross-cultural projects

This paper considers the issue of evaluation in cross-cultural contexts, where the focus on Western principles of technology quality can be problematic. We present a case study of designing a Digital Community Noticeboard with an Australian Aboriginal community, and discuss how its evaluation may be better approached in terms of 'designing futures'. Rather than viewing evaluation as the validation of a designed artefact, our approach is focused on mapping progress towards a vision for a future world that shapes, and is shaped by the noticeboard, as the two have mutually evolved. The resulting insights go beyond the technical and lead to a better understanding of how the noticeboard may fit with everyday practices, local values and social protocols, and issues of sustainability. This shift in perspective helps to foreground these socio-technical factors and research problems that may otherwise pass unnoticed.