The Effect of Sample Size and Test Length on Equated Scores and Error of Equating: The Case of Iranian National Tests

The purpose of this research was to assess the effect of sample size and test length on the equated scores and equating error of Kernel method (using different methods of chain and poststratification smoothing) as well as the merits and demerits of this method compared to classical equating techniques. Therefore, the population and sample participants were those who took part in Iranian National Tests (TOLIMO, Comprehensive Tests of Iran Educational Testing Service) administered in 2012-2013. TOLIMO had a number of 123 items including 17 anchor tests in each form. To analyze data collected from Comprehensive Tests of Iran Educational Testing Service, only those items related to common general-domain subjects of mathematics and physics, science and humanities were utilized. To investigate the effect of sample size on the accuracy of equating the above mentioned tests, three samples of 200, 500, and 1000 people were randomly selected from among data collected from all participants and analyzed. A 40-item sample (10 items from each subject) was randomly chosen from general subjects of comprehensive tests to examine the effect of test length on the accuracy of the results of equating. Thus, in comprehensive tests, two 100-item and 40 items sample tests were analyzed with samples of different sizes. The proper equating design in TOLIMO was NEAT design, but in Comprehensive Tests it was homogeneous groups design. Equating methods in the respective tests have been mean, linear, equipercentile, Circle arc and Kernel methods. On the whole, the larger the examinees sample whose scores were taken into account in the analyses was, the lower standard error of equating would be. The findings also showed that whenever there was an increase in both sample size and test length, an improvement was observed in the fitness related to Kernel smoothing as well. Generally, with small sample sizes, Kernel method is more advantageous than other methods of classical equating.