Using Generalizability Theory in Management Research with an Emphasis on Job Analysis

Studies on management with the subject of evaluating employees’ performance, assessing employees’ job competencies and job analysis from assessors are used as a source of data collection. The results can be used when the reliability of data is acceptable. Researchers use inter-rater reliability of Cronbach’s Alpha to estimate the reliability of data. The formula considers errors in data as random error. Systematic errors in most research condition must be considered when their reliability is estimated. For this reason, these formulas can’t estimate the reliability assessment of assessors properly. This article showed the application of generalized theory to predict the reliability assessment of assessors. At first, the weakness of classical test theory and its ability to estimate the reliability generalized theory was analyzed. Then, it explained examples to show the users the steps for applying generalized theory with crossed and nested measurement models which researchers can estimate job analysis reliability of data more accurately.