A framework for the acceptance testing of geospatial search engines

Geospatial search engines are an essential component of spatial data infrastructures and enable a broad spectrum of environmental applications. The back-end implementation of these search engines has evolved from traditional text-based information retrieval systems into more specialised search engines. However, to assess the actual improvement brought by this evolution, thorough testing is needed. The aim of this work is to propose a framework for the acceptance testing of geospatial search engines that assesses their functionality, effectiveness, and user-friendliness. For each quality attribute, the framework proposes different testing design techniques and guidelines for their practical implementation. To demonstrate its feasibility, it has been applied to the evaluation of a geospatial semantic search engine of the Spanish National Geographic Institute. The evaluated search engine showed a sufficient level of functionality and effectiveness. However, the usability results were barely satisfactory due to perceived problems associated with complexity, inconsistency, and low learnability.