Detecting Emotion as a Construct Irrelevant Factor in Reading Section of TOEFL

Understanding the impact of various factors on language testing is important. Therefore, it is necessary to understand how they affect test scores in order to design and standardize language tests (Bachman, 1990). Based on the same logic and considering the need of fair reviewing for any tests claimed by ETS (2010), it is essential to identify, reduce and eliminate factors unrelated to the structure that hinder the optimal performance of test takers (Messick, 1989). According to Vinson (2014), words themselves are a powerful tool for expressing emotions. Does a factor such as vocabularies in a text stimulate emotional reactions? When we read a text, we use our knowledge to understand its vocabularies, but in addition, by reading the vocabularies, emotion may be stimulated, which are deliberately not examined during the test. This research is intended to investigate if the vocabularies of TOEFL passages are likely to stimulate emotion as the construct-irrelevant factor which could affect the accuracy and legitimacy of the TOEFL test. Also, our hypothesis is that the amount of emotion evoked in the words of TOEFL texts is different from each other, and other factors intervene in this category. By using the initial pilot sampling and with the help of PASS Software to determine the size of the final sample, 393 people were randomly selected by Random Number Generator Software. In addition, according to the method of detecting emotions by forming an emotional dictionry proposed by Turney (2002), the present study labled parts of speech of each word and then the words were grouped together as meaningful expressions into a five-page list of phrases like a dictionary. Through this method, the present study could determine the intensity and valenance of the selected particpants’ emotions in relation to the phrases selected from TOEFL iBT reading passages. Also, it was significant to measure if emotional intelligence could be influential on the evoked emotions of the words and phrases; thus, Emotional Intelligence Questionare of Schutte (1998) was selected. The research procedure was that the selected participants read three TOEFL passages without answering their reading questions; instead, they did Emotional Intelligence Questionnaire and self-reported their emotions through the five-page list of words and phrases. The results of this study confirmed the validity of the research hypotheses, in the sense that not only the words and phrases of the three TOEFL passages in this research caused emotions but also the three passages were different in terms of emotion, and it can be argued that this can be a construct-irrelevant factor when reading and comprehending TOEFL passages. The results of this study can increase the awareness of TOEFL test designers. In other words, TOEFL test designers must consider the effect of emotional elements in language assessment because these elements may disrupt the mental order of test takers and can affect their performances. This study casts doubt on the validity and reliability of the TOEFL as a standard test. ETS (2010) is interested in a fair review of tests to identify and reduce factors unrelated to the structure so it is useful considering the emotional interactions in the process of assessing the validity and reliability of any tests.