Analysis of the Lived Experience of University Professors from Evaluating the Students' Learning Outputs

In the 21st century, the importance of the higher education system in any society has become so critical that, according to Malcolm Gillis, today the fact that a nation be rich or poor greatly depends on the quality of higher education. But the hasty quantitative-oriented approach caused it to remain only within the limits of directives and regulations, and the qualitative dimensions of education were considered less. In the meantime, paying attention to outputs- which is equivalent to outcome, yield and results- and evaluation became more important because evaluation is part of the learning process, and the three streams of training, learning, and evaluation work together. Accodingly in 1985, Rabart Gagne formulated the theory of education based on cognitive principles which includes the learning conditions or the circumstances in which learning takes place. There are two crucial steps in applying this theory: The first step is to determine the type of learning output. Gagne has determined the five main outputs. The second step is to identify learning events or factors that make a difference in learning. In defective educational systems, the appraisal criterion is basically observable behavior and large-scale classroom exams with the use of achievement tests,and the goal is passing semesters and graduation,and learning actually does not take place.Consequently, when evaluation is in the service of learning, the learner has a deep understanding of his/her performance and evaluates his/her learning continuously and considers his/her effort-and not random or external factors-as the factor of his/her success. This issue and its importance is studied and discussed by many researchers, e.g.(Klein et al.,2018); (Melgozzo et al.,2017); (Sumble,2016); (Havnes et al.,2016); (Klimova,2014); and (Naseri ,2017). By examining their findings and considering the problems and obstacles, and insufficiancy of facilities in universities, including lack of appropriate and different tools in evaluating the learning outputs of students, especially in the field of humanities, in which the theoretical aspect, score and exam sheet is the criterion for evaluation, the researcher faced this question: What are the dimensions, components and evaluation indicators of the students’ learning outputs? Research

The present study examined the "analysis of the lived experience of university professors from evaluating the students' learning outputs".The research purpose is practical and research field includes all professors in the field of educational science in universities of Tehran province. In the qualitative stage, the snowball sampling method was used. Also, by continuously comparing the data until reaching the theoretical saturation, a semi-structured interview was conducted with 10professors.In the quantitative stage, using the Cochran's formula,193 out of 387of these professors were surveyed by simple random sampling. The tools for collecting data at qualitative stage are semi-structured interviews, field studies and library studies, and in the quantitave stage, a researcher-made questionnaire based on the five-scale Likert was designed to evaluate qualitative data. To examine the content validity, triangulation of data sources was used and professors who were interviewed as well as advisors and readers were also surveyed, and with the help of a questionnaire measured with Cronbach's alpha tool, the idea of other professors was applied to assess the reliability. In terms of method and manner of implementation, the research is a mixed (exploratory) method. In the qualitative stage, the phenomenological method based on Van Manen’s seven-scale model was used, and in the quantitative stage, to validate the model, the Confirmatory Factor Analysis and the Structural Equation Model were used. Data normality was assessed by K-S test and components were assessed by one-sample T-test. Statistical analyzes were performed by SPSS and Lisrel softwares, and the final model was extracted. Research

Here, the research questions are answered in two stages, qualitatively and quantitatively. General Question: What are the dimensions, components, and evaluation indicators of students’ learning outputs? First, to do the analysis, the whole transcripts of 10 interviews and document surveys between 2012 and 2020 were divided into 1405 initial codes, and after several screenings and continuous comparisons, 320 open codes were recorded.To follow the inductive path, the initial codes were classified into 87 sub-categories and 17 main categories (dimensions) and were subdivided into 5 constructs. Question 1-What are the characteristics of a learning output evaluation model? Constructs, dimensions, categories and indicators after review by professors were categorized as follows.  Table (1):5 extracted constructs and subdivisions No. of Indicators with Higher Priority According to Professors' Survey No. of Dimenstions No. of Catatories No. of Indicators Constructs 60 indicators 134 38 6 The nature of evaluation 10 indicators 28 9 3 The nature of the curriculum 45 indicators 70 17 3 Evaluator (professor) 28 indicators 51 14 3 The nature of learning outcomes 26 indicators 37 8 2 Evaluatee (student)   Question 2-What is the model for evaluating learning outputs? From the results of the qualitative part, a model was extracted indicating the evaluation of learning outputs is surrounded by 5 constructs, and each construct is connected to a number of dimensions, categories and indicators. Question 3-To what extent is the mentioned model valid? In the quantitative step to assess the validity of the model extracted in the qualitative part,at first by using central indicators, the scattering of variables were described, the normality of the data was checked, and the correlation matrix with 5constructs was defined as the input for Lisrel software. Factor Analysis Test was performed on (components of) each construct. As a result of standard parameter estimates, Chi-Square and RMSEA indices showed that the output of pattern fitting indices is statistically significant in each construct and all factor loads were accepted and prioritized due to being at a high level.The amount of Chi-Square in the degree of freedom of the models and the values of one-sample T-test showed that direct relationships exist. Therefore, the null hypothesis was rejected with 95% confidence for these components, and the research hypothesis was confirmed.

Finally,according to the prevalent conditions of Iranian higher education system and the lived experience of university professors, there was an attempt to design a model that depicts the evaluation of learning outputs in 5 constructs, 17 dimensions, 87 categories and 320 indicators that are all related. According to the professors, paying attention to all of them will have a positive effect on the evaluation of learning outputs since in the evaluation process, the role of the student, the professor, and the curriculum are influential, and disregarding each of them can change the output.These findings are consistent with the results of studies by (Klein et al., 2018); (Melgozzo et al., 2017); (Coates, 2016); (Paulini, 2015); (Alkuran,2012); and (Sufizadeh,2015). Therefore, we hope to see the improvement of the quality and effectiveness of the national higher education system by using the results of this study and other related researches.