Psychometrics of Questionnaires Designed for the Evaluation of Safety Levels and Safety Culture Case Study: Power Plant Generator Manufacturing Company

Abstract

The psychometrics of instruments in safety performance evaluation is essential for the accreditation of an organization’s safety evaluation and has been emphasized in many studies. Psychometrics pertains to the validity and reliability of an evaluation instrument and describes its precision and consistency. The aim of the present study was to evaluate the psychometrics of safety level and safety culture questionnaires to provide a reliable and valid instrument for safety performance evaluation in industries. This descriptive cross sectional study was conducted with the intention of psychometric instruments used to evaluate the safety level and the safety climate at the Mapna Pars power plant generator manufacturing and engineering company. The face validity of the questionnaires was analyzed quantitatively and qualitatively. In the quantitative method, the importance of the influence score was applied and the content validity was calculated using the Lawshe method. Content Validity Ratio (CVR) and Content Validity Index (CVI) were also used. For this purpose, elicitation was obtained from experts within the professional health and safety community. The reliability of the instruments was determined via the Cronbach's alpha test and the Interclass Correlation Coefficient (ICC) test. Finally, the standardized questionnaires were used to evaluate the safety level and safety climate of the industry as a case study. The obtained data were analyzed using SPSS software solution v20.  Based on the quantitative face validity results obtained at the Mapna Pars Company, one question was removed from each of the final instruments. The content reliability analysis revealed that the safety level questionnaire had a CRI of 0.99, CVR of 0.91, and was accepted. The safety climate questionnaire had an acceptable CVI of 0.95 and an acceptable CVR of 0.82. Regarding the reliability analysis, a Cronbach's alpha of 0.72 and 0.89 was obtained for the safety level questionnaire and the safety climate questionnaire, respectively. It can be concluded that both questionnaires had an acceptable level of internal consistency. The re-application of the questionnaires after two weeks revealed a relatively consistent safety level (ICC=0.90) and safety climate (ICC=0.74).  Analyzing the data obtained in the present study showed that the safety level questionnaire with 66 questions and the safety climate questionnaire with 93 questions had acceptable validity and reliability. Thus, it may provide a useful approach for safety evaluations in similar industries.