An Overview of the Generalizability Theory and its Implications in Medical Education

All measurements, including medical students’ assessment, are potentially prone to error from various sources. Considering the importance of maintaining the quality of assessment methods in medical education, finding methods to estimate the error and to decrease it, is essential. Many developments have been achieved in measurement theories during recent years, which can be used in this way.
In this study, following reviewing the classical test theory (CTT) and its limitations, we discuss the generalizability theory (GT) using some examples of the Objective Structured Clinical Examination.
According to the CTT, students’ scores in exams are not their true scores and are accompanied with some levels of error. Determining reliability of tests helps us to assess how much the random error has been affected the score. In the CTT, it is impossible to consider all error sources simultaneously. To cope with this limitation, the GT specifies share of various sources of error based on the ANOVA models. This theory estimates reliability of tests and offers some recommendations for improving their generalizability.
In CTT, specification of effect of various sources of errors is not possible, whereas GT can be used to identify share of each error source. Hence, the test reliability coefficient can be estimated and also a more reliable test may be planned.