Evaluation of an implementation guide for an IT standard using surveys

Abstract

This manuscript describes how we evaluated an implementation guide designed to help public organizations in Costa Rica that must implement an IT standard. In CLEI 2011 we described how we created the guide and how we used the guide in one financial entity. In this paper, we continue this effort by describing how we evaluated the quality of the designed implementation guide. To evaluate our product, we relied created surveys that were completed by a group of experts. The sample (nine experts in total) consisted of IT experts both from the entity that designed the IT standard as well as experts from the financial organization that participated of the implementation process of the standard. Therefore, our evaluation takes into consideration the opinion of both designers and implementers of the standard. Our results show that both groups of experts consider that our guide simplifies the implementation and appraisal processes. A majority of experts also believe that it is easier to understand the standard using our guide and that an implementation team would be more efficient if the guide is used.