Setting Standard Threshold Scores for an Objective Structured Clinical Examination using Angoff Method and Assessing the Impact of Reality Chacking and Discussion on Actual Scores

A variety of standard setting methods are used worldwide for medical examination acceptance scores while standards of most exams in our country are pre-determined fixed scores which are set without any scientific methodolgy. The aim of this study is to determine minimum pass level for a pre-internship objective structured clinical examination using Angoff method in Tehran University of Medical Sciences. After designing the questions for examination, a panel of eleven faculty members was formed. These judges were asked to individually estimate the probability that a borderline student would pass each station. The mean of all stations estimated by judges was considered as the standard for the whole exam. This procedure was repeated twice more after sessions of discussion between judges and checking students’ real scores. The individual standard for the whole test was 49.15 while it turned to 49.90 after discussion and finally 51.52 after checking the real scores of students. The change of standard of the whole test after checking real scores was significant compared to individual standard (p=0.02). It showed no significant difference compared to the second standard. The rates of passing students according to the three standards were respectively 67.6%, 64.8% and 58.1% which showed a significant reduction in the third compared to the first one. Angoff method was used in this study to set standard for an OSCE. According to the findings of the study, it seems to be a credible and reliable procedure, especially when group discussion and reality check are used.