فصلنامه مطالعات اندازه گیری و ارزشیابی آموزشی
پیاپی 30 (تابستان 1399)

The aim of this study was to investigate the invariance of the factor structure of the national test and the item parameters (factor and threshold) in the provinces. The research method is descriptive correlation. In order to investigate the purpose of the research, a sample of 1000 participants in the math test of the Mathematics and Technology experimental group of 1396 was selected from each province. The results showed that in most items, the performance of the examinees in Tehran, Isfahan, Khorasan Razavi, Fars, Mazandaran, Yazd and Alborz is better than most provinces. The number of non-invarince items in Tehran and Ilam is higher than other provinces. The alignment optimization method showed that 37.5% of the items in the first factor and 16% in the second factor for all provinces had approximate invariance of thresholds and 83% of the items in the first factor and 71% in the second factor had invariance in factor loadings. The number of non-invariance items in the second factor is higher than the first factor, so it is better to use the first factor items to compare the examinees in the provinces. Investigating DIF by using the alignment method alone is not enough in a large number of groups. If a specific group is involved in the study, it is necessary to use two-group DIF methods.

The aim of this study was to investigate the role of employers in the evaluation of curricula of the top universities of the country and the world in order to design a scale for employers. The research method has been a sequential exploratory mixed method of instrumentation types and the statistical population consisted of six groups. The first group is comprised of faculty members and experts in curriculum development of the top universities. The second group includes officials and educational planning centers of the top universities. The third group consists of relevant officials of the Ministry of Science, and the fourth group is the world’s top universities. Also, the fifth group involves instructors of master's degrees in educational sciences, social sciences and administrative sciences from public universities, and the sixth group consists of employers. Besides, data assimilation tools, semi-organized interviews, document analysis, and researcher-made scales are used in the research process. The findings show that in the world’s top universities, there is a close relationship between the university and the employers. Employers have roles such as helping to develop curricula, setting professional standards for jobs, assessing students' knowledge and skills, accreditation of curricula, explaining expectations to graduates, job counseling, and even teaching. In comparison, in the country's top universities, employers, except for a few cases, do not participate in the process of developing and evaluating curricula. By using the data, a curriculum evaluation scale has been developed from the opinion of employers, including 21 phrases and six open-ended questions.

The purpose of this study was to develop a contextual framework for the behavioral requirements of university faculty members by exploring the themes and meta-themes of their teaching roles. The research method was phenomenology and the research field were students of postgraduate courses in the humanities at Ferdowsi University of Mashhad. The purposive sampling of maximum variation type was performed. Data collection was done by semi-structured interviewing technique with participants and finally saturated in thirty-two interviews. The results of the data analysis by Colaizzi's method showed that the framework of teaching behaviors of faculty members from the viewpoint of postgraduate students has 15 thematic and 5 meta-thematic including teaching expertise, coaching, fair assessment, empowerment and accountability

The aim of this study was to identify the requirements for optimal evaluation of the performance of primary school teachers.  The research method was based on theoretical inquiry and the resources of data included documents, the opinions of the experts, the global experiences, and the views of the beneficiary teachers. In order to answer the questions, two methods of thematic analysis and document analysis were used, and nine teachers' evaluation requirements were extracted. Then, the components for purposeful evaluation of the performance of elementary school teachers in four areas of communication with students, professional development, personality characteristics and social interaction were identified. Based on the alignment of the information gathered through different sources and the general consensus of experts and their alignment with the findings, the overall validity of the research process and the framework of assessment implications were confirmed. Based on the findings, enhancing the quality of education in the area under study requires policy makers to change their evaluation components and should involve a variety of sources including peers, parents and students to be used. It is also necessary to build on the real expectations of teachers by the general public and stakeholders, and use various valuation tools and sheets. If the existing procedure in teacher evaluation continues and little attention is paid to its requirements, the growth of the education system will be hampered.

The quality of a PhD candidate’s performance is the most important factor that can affect admission committee members’ decision-making and selection of applicants. One of the issues that has always accompanied candidates’ scores  in admission exams is the validity of these scores in predicting the candidates’ successful performance . The primary purpose of this research was evaluating the predictive criterion validity of subject tests scores  in four engineering PhDprograms on interview measures  as the criterion for successful performance . In this descriptive correlational study, candidates’ scores in subject tests  and interviews  in four engineering PhD programs in 2018 were analyzed based on the Generalizability theory. Depending on their majors, examinees’ data included their scores on 3 to 8 subtests in subject tests and 2 composite interview criteria . Data were analyzed using mGENOVA software based on a multivariate analysis with a single facet  design. The relationship between two composite universe scores was illustrated by the bivariate normal distribution in a typical scatter diagram. The results show that subject tests scores can predict candidates’ scores on the interview criterion as an index for successful performance except in the field of Information Technology. The large number of subtests in subject tests,their instability, and the construct irrelevant variance were among the factors that may influence the accuracy of predictions, especially on the cut scores, for accepting or rejecting candidates. Accurately predicting the success of an applicant’s performance  based on test scores   is an unattainable goal. However, precisely defining the criteria for success  in PhD programs, increasing the significance of general English and aptitude tests, adjusting the difficulty level of tests, reducing measurement error in predictive measures  , having a two-step assessment program in which the general language and aptitude tests are administered as a primary screening by NOET and the subject tests and interviews by universities can be effective method to increase the criterion validity of test scores in predicting candidates’ successful performance and enhance the chances of admitting more qualified students.

The purpose of this study was to develop and validate the quality assessment framework of massive open online courses. To achieve the purpose, the factors affecting the quality of MOOCs were extracted, classified and formulated as a framework. The research population comprised of educational technology experts across the country. The sample (200 n) were selected by convenience sampling technique due to the dispersion of the population. A researcher-made questionnaire was designed, validity and reliability of which was confirmed by the educational technology experts. The research data analyzed using confirmatory factor analysis method and SmartPLS software. The results indicate the MOOC quality assessment framework based on the Cronbach's alpha measures (composite reliability, convergent validity, and divergent validity) is desirable, suitable and robust. This framework is also robust in terms of the structural model measured by the R2 and Q2 indexes.

Equating is one of the most important issues in educational measurement, violation of the assumptions of which results in serious challenges in it. In multidimensional tests, the use of unidimensional equating methods causes bias in results. Therefore, the purpose of this study was to identify the optimal equating methods in multidimensional tests. Six equating methods consisting of unidimensional and multidimensional methods was compared with each other. The equipercentile method was considered as the criterion for comparing the other methods in terms of being robust against the unidimensionality assumption. The statistical population consisted of all the candidates in Mathematics’ entrance exam in the years 2017 and 2018. Mathematics’ exam data of 5,000 people from both years were selected for equating. Test dimensions were determined using NOHARM and MPLUS software, the ltm package was used to estimate the unidimensional parameters and the mirt package in the R software was used to estimate the multidimensional parameters. IRT unidimensional observed score and true score equating conduct with PIE program and equipercentile equating was performed using R equate package. The mirt, mvnorm, and MASS packages in R software were also used for multidimensional equating.  The results showed that the most optimal method for multidimensional tests equating was FULL MIRT observed score equating,and then the unidimensionalized MIRT observed score equating is the appropriate method and that using unidimensional methods of the observed score and true score are not efficient in such conditions. Therefore, it is recommended that FULL MIRT observed score methods are used in equating tests with multidimensional structure.