The Psychometrics of Questionnaires Designed for the Evaluation of Safety Levels and Safety Culture

The psychometrics of instruments used for safety performance evaluation is essential for the accreditation of an organization’s safety evaluation and has been emphasized in many studies. Psychometrics pertains to the validity and reliability of an evaluation instrument and describes its precision and consistency. The aim of the present study is the psychometrics of safety level and safety culture questionnaires to provide a reliable and valid instrument for safety performance evaluation in industries.

This descriptive-cross sectional study, was conducted with the intention of psychometric instruments used to evaluate the level of safety and the climate of safety in the Mapna Pars power plant generator manufacturing and engineering company. The face validity of the questionnaires was analyzed quantitatively and qualitatively. In the quantitative method, the importance of the influence score was applied and content validity was calculated using the Lawshe method. Content Validity Ratio (CVR) and Content Validity Index (CVI) were also used. For this purpose, elicitation was obtained from experts within the professional health and safety community. The reliability of the instruments was determined via the Cronbach's alpha test and the Interclass Correlation Coefficient (ICC) test. Finally, the standardized questionnaires were used to evaluate the safety level and safety climate of the industry chosen as the case study. The data obtained were analyzed using the SPSS software solution v20.

Based on the quantitative face validity results obtained at the Mapna Pars company, one question was removed from each of the final instruments. The content reliability analysis revealed that the safety level questionnaire had a CRI of 0.99 and a CVR of 0.91 and is thus acceptable. The safety climate questionnaire had an acceptable CVI of 0.95 and an acceptable CVR of 0.82. Regarding reliability analysis, a Cronbach's alpha of 0.72 and 0.89 was obtained for the safety level questionnaire and the safety climate questionnaire respectively. This indicates an acceptable level of internal consistency for both questionnaires. The re-application of the questionnaires after two weeks revealed a relatively consistent safety level (ICC=0.90) and safety climate (ICC=0.74).

Analyzing the data obtained in the present study shows that the safety level questionnaire with 66 questions and the safety climate questionnaire with 93 questions have suitable validity and reliability and can be used for safety evaluations in similar industries.