Investigating the Trend of Changes in the Quality of Objective Structured Clinical Examination to Assess the Clinical Competence of Medical Students in Tehran University of Medical Sciences

Considering that in recent years the clinical competency test is performed to measure the performance of medical students before graduation, it is necessary to study its psychometric properties in order to make managerial decisions to hold a valid and reliable test.

In this retrospective study, all medical students of Tehran University of Medical Sciences who participated in the clinical competency test during the academic year 2017-2018 were included in the study. Participants' scores and the total score of the OSCE test were collected and analyzed separately for each station. In order to evaluate the quality of OSCE, Difficulty Index, Discrimination Index, R2 and differentiation of degrees were calculated using SPSS software. The reliability of the test was determined using Cronbach's alpha. The correlation between the total grade point average of the students was calculated by the final OSCE score.

772 interns were included in the study in the form of 4 courses of competency test. The value of R2 for each test is 0.33 to 0.78%, 0.75 to 0.91%, 0.63 to 0.90%, 0.53 to 0.89%, respectively. The Cronbach's alpha test was obtained from 0.52 to 0.76. The range of Discrimination Index for each test was equal to -0.02 to 0.52, -0.09 to 0.64, -0.05 to 0.64, and 0.94 to 1 respectively. The range of Difficulty Index for each test was 0.68 to 0.93 and 0.21 to 0.88, 0.62 to 0.91, and 0.72 to 0.96, respectively. The correlation coefficient between the total grade point average and the final OSCE score was calculated to be 0.56 (P<0.001).

The internal consistency of the tests was acceptable. The R-index varied at different test times at different stations. The range of Difficulty Index and Discrimination Index in different test times have changes that can be resulting from the educational effects of test implementation for students and improving the design quality of stations. There was a direct and positive relationship between students' GPA and OSCE final score based on Pearson correlation coefficient.