Analyzing the Quality of Objective Structured Clinical Examination in Alborz University of Medical Sciences

One of the evaluation methods is based on the performance of objective structured clinical examination. Passing this exam is necessary for medical students to graduate, practice medicine and participate in a specialized course. This study is designed to analyze the quality of these tests using psychometric indicators.

In this cross-sectional descriptive study, the test of three periods of clinical competence, respectively, were analyzed. Average (standard deviation), the cut-off score was (by borderline regression method), rejection rate, R2 determination coefficient, root mean square error to estimate the the cut-off score, difficulty coefficient, neatness coefficient were calculated in individual stations and the whole test.

64 medical students along with 26 expert examiners from each station were present in this exam. According to the cut-off score in the 31st exam, all students got a passing grade and we did not fail. The total RMSE was equal to 0.55 and the value of R2 in different stations was obtained from 0.46 to 0.85. The hardest and easiest stations based on the difficulty coefficient were gynecology, social medicine and orthopedics respectively. In the 32nd exam, the rejection rate was one person (0.23%). The total RMSE was 0.57 and the value of R2 in different stations was obtained from 0.38 to 0.87. The most difficult and the easiest stations based on the difficulty coefficient were radilogy, emergency medicine and surgery respectively. In the 33rd exam, all students got a passing grade. Total RMSE was equal to 0.76 and R2 value was obtained in different stations from 0.39 to 0.95. The most difficult and easiest stations based on the difficulty coefficient were internal glands and surgery respectively.

To improve the quality of the OSCE exam, measuring its quality is vital. The use of several psychometric indicators can be an important step in determining the quality of the test. The results obtained during the three test periods indicate that the quality of the test is acceptable in most of the stations and some cases need to be reviewed and improved.