The effect of geographical clustering on parameter estimation and ranking of individuals in large scale assessments

The aim of this study was to investigate the effect of geographical clustering on the estimation of ability parameter (before ranking) as well as the ranking of examinees in large-scale tests (such as national university entrance examination). The design of this study can be considered as survey and because of analyzing the data previously collected by the National Organization Educational Testing (NOET), this project can also be considered as a secondary data analysis. The statistical population of this project includes all the candidates of the Mathematics group who have participated in the national exam of 2013-2014. The sample used in this study includes data on 3,000 examinees in the mathematics group from regions (quota) one, two and three, that have been provided to researchers by the NOET. The main data used in this study are the examinees’ scored responses to test items (1 for a correct answer and 0 for an incorrect answer) and the code related to the selected quota. In summary, it can be concluded that differences between the three regions, or any type of clustering, is considerable when the level of analysis is the overall sample and beyond clusters. In such a situation, it is possible to observe a significant difference between rankings using different methods, even when the value of intra-class correlation (ICC) is very low (like this study).