فصلنامه مطالعات اندازه گیری و ارزشیابی آموزشی
پیاپی 42 (تابستان 1402)

The aim of this study was to compare the accuracy and measurement error of dichotomous and Polytomous IRT models in scoring high-stakes, large-scale ability tests.

The statistical population of this study was included all the participants of the lawyer's license external tests in 2016 and 2018, from which 5000 persons and 5000 persons respectively were selected by random sampling. In addition, data collection was done using the responses of the participants of the above exam. Accordingly, the research method is experimental.

The analysis of the findings showed that among the dichotomous IRT logistic models, the 3-parameter model, and among the nominal Polytomous models studied, the 3-parameter model are a better fits and information compared with other models on the data under study.

Considering the more favorable fit and the level of information of the 3-parameter dichotomous model and the 3-parameter Polytomous model compared with other models, the use of these models in scoring can increase the accuracy of measurement and reduce the error. In addition, the use of these models also helps the fairness of the selection process of the applicants for the lawyer's license exam.

The present study was done in order to examine the severity/leniency and the central tendency level of raters in scoring of performance tests performed by National Organization for Educational Testing (NOET).

For this purpose, the secondary data in Sketch Architecture Test (1396 and 1397 solar, respectively with 5437 and 7459 people), Industrial design test (1396 solar, 1365 people), Music recognition test (1397 solar, 569 people), playwriting test (1396 solar, 97 people). The data were analyzed by classical methods and many-faceted Rasch models and the results extracted.

The results from classical methods show that in both Sketch Architecture Tests, raters’ consistency is generally acceptable, but in other tests (Industrial design, music cognition and playwriting) homogeneity is low. Raters’ consensus is low in all five examined tests. Results from many-facet Rasch models show that in both Sketch Architecture Tests rater's severity and use of lower scores of rating scale effects are present, but as expected, there was not any effect for central tendency. Unfortunately, due to the nature of incorrect data collection designs in Industrial design, music cognition and playwriting tests analysis with many-facet Rasch models was not possible.

Based on findings it is recommended that when global performance tests are evaluated by the NOET organization raters, firstly; the proper design for evaluating have been selected and used, and secondly; to prevent the effect of severity or leniency and agreement (consensus) reduction between raters, the training of them for scoring performance tests to be considered

Nowadays, cognitive diagnostic assessment (CDA) is highly attended due to attain deeply information about examinees. In present study, CDA is used to provide more information about science responses of Iranian fourth graders who participated in TIMSS 2015.

11 science attributes were identified and the Q matrix was created. Then, responses of 3823 students to 206 items was analyzed by G-DINA model.

The results showed that the model can produce suitable diagnostic evidences. Also, three attributes including: Recognize, Providing examples, and Describe had higher attribute probabilities than others and their mastery probabilities were more than 0.50. Whereas, only probability of Recognize was higher than 0.65 and this is considered as mastered attribute. In other hand, Predict and Scientific inquiry had lowest attribute probabilities.

In general, the results revealed that there are sizable deference between knowledge-based attributes and higher-order thinking attributes in students, that can associated to distance between intended and implemented curricular.

The main goal of this paper is to present a multi-level model for analyzing the scores of the national entrance exam to universities and higher education institutions.Based on its objective, this research is an applied and a descriptive research for analyzing and explaining the factors influencing the total score of the candidates participating in the university entrance exam. We used data of the 2019 year exam which include information on socio-economic status of the candidates' including educational, health, cultural and economic variables at individual level as well as at the level of counties and provinces of the country and We used the HLM software to stimate the multilevel models.The estimation of the three-level model showed that 94.6% of the changes in the total test score were due to the differences in the individual and family characteristics of the candidates. 3.6% of the changes in the total score were related to the educational, health and cultural factors of the city and 1.8% to the economic participation rate of the province. Cosidering that 5.4% of the changes in the total score are related to the geographical location, it can be acknowledged that the difference in the total score of the candidates is not only rooted in their socio-economic status, but the simultaneous effect of this situation with the level of prosperity of the cities from the perspective of educational, health, economic and cultural factors also contributes to the intensification of the difference in the total score.

The total purpose of this research was to analyze the content of various articles related to curriculum evaluation patterns based on emphasis on curriculum elements and internal and external effectiveness criteria.  quantitative, descriptive and based on quantitative content analysis. The statistical population is all research sources published in the field of curriculum evaluation, and all of them were selected as samples in the five-year period from 2017 to 2022 and 2016 to 2016. Documentary measurement tools and data collection methods were based on curriculum evaluation patterns and data analysis was done using Shannon's entropy technique. Among ten elements, content had the highest and most effective importance coefficient and learning time had the lowest importance coefficient. In internal and external evaluation criteria, logic had the highest importance coefficient and coherence had the lowest importance coefficient. Also, among the types of curriculum evaluation models that were used to analyze the content of the selected articles, Kirkpatrick's model was the most effective and had the highest importance coefficient, and Tien's model had the lowest importance coefficient. In order to ensure the correct implementation of a curriculum and the improvement and promotion of curricula in the field of education, it is necessary that the effectiveness of all curriculum elements and the internal and external criteria of the curriculum should be considered in the evaluation analysis of articles.

The current research was conducted with the aim of comparative study of decision making theories and educational career choice and in order to provide a strategy for evaluating and improving the educational career guidance program of Iran's education system.  This research was done with a comparative approach and with an analytical documentary method. Therefore, the theoretical documents were collected using related keywords and digital libraries and online databases and were analyzed through the researcher's flash capture and data summarization and classification techniques. In order to check the validity of the findings, the researcher's self review criteria were used during the process of data collection and analysis, as well as the development and rich description of the data to ensure transferability. The effective factors in educational career guidance based on the presented theories can be categorized into three parts, including individual factors, social factors and environmental social factors. The results of the evaluation of Iran's educational career guidance program with the statistical classification also showed that, despite paying attention to some of these factors, the educational career guidance program has not been very successful in implementation. The effectiveness of educational career guidance programs requires identifying and considering all the influencing factors in this field. In this context, by designing and implementing the educational career guidance curriculum, all the factors and criteria that have been proposed in the theoretical base as important factors in choosing an educational career can be taken into account during the process and systematic steps in the entire educational levels of the students.