The Construct Validity and Predictive Validity of In-Training Evaluations of Emergency Medicine Residents in Tehran University of Medical Sciences

The evaluation of psychometric indicators in order to evaluate the quality of academic achievement exams is of particular importance. The aim of this study was to investigate the results of the written exams and internal evaluations among the emergency medicine residents in terms of the construct validity and also their ability in predicting resident’s academic success.

In this descriptive correlational study all the emergency medicine residents of Tehran University of Medical Sciences who could enter the emergency medicine residency program within 2009 to 2014 and had participated in the pre-board exam till 2017, were selected through the census method (n=199). The certification exam score was considered as the main variable; then, correlation of written and internal evaluations results were compared with it. The Pearson correlation test was used to determine the correlation between the scores.

Finally, the evaluation results of 125 residents were analyzed. The correlation between first and second internal evaluation scores, first and second written scores with the certification exam scores were (p=0.006, r=0.3), (p=0.014, r=0.2), (p=0.202, r=0.1) and (p=0.000, r=0.3), respectively. The mean scores of the first, second and third year residents in written exams were 77.92±12.57, 91.33±8.05, 123.13±8.52 out of 150 in 2014; the mean of these scores were 67.44±8.52, 73.17±23.41, 121.52±6.38 in 2015 and 68.71±28.04, 88.24±14.34 and 118.15±7.73 in 2016, respectively

The results of this study showed that in-training evaluations of the emergency medicine residents had acceptable predictive validity in predicting academic success of residents. Also, the ability of the written exams in discriminating between residents of different grades demonstrated the construct validity of these exams.