فهرست مطالب

International Journal of Language Testing
Volume:2 Issue: 1, Mar 2012

  • تاریخ انتشار: 1391/10/11
  • تعداد عناوین: 4
|
  • Payman Vafaee, Nesrine Basheer, Reese Heitner Page 1
    An important assumption in language testing is that test items or observable variables tap the underlying latent traits hypothesized in the theoretical model or constructs governing the design of the testing instrument (e.g., Shin, 2005). Accordingly, the present study sought to investigate the extent to which scores from the grammar sub-test of the Columbia University Community English Program (CEP) placement test could be interpreted as indicators of test takers’ grammatical knowledge. The authors adopted Pupura’s (2004) theoretical model of grammatical knowledge, which hypothesizes that grammatical knowledge consists of two underlying traits of form and meaning. To this end, the authors conducted a confirmatory factor analysis (CFA) to investigate whether there is a match between the CEP grammar test data (n= 144) and the theoretical model as hypothesized. Since the test items were not discrete point but were nested within one of four tasks (each with their own theme), by endorsing the interactionist view of construct definition, effects of these four themes (context) on individual items were also investigated. A multitrait-multimethod matrix (MTMM) model achieved the best possible model fit based on substantive and parsimony considerations. It included two underlying traits of grammatical form and meaning and four method (context) factors, and confirmed that the CEP test examined the grammatical knowledge and included the effect of context as a part of its construct. These findings support the interpretive argument presented for the construct validly of the CEP grammar test, and the appropriateness of the explanation inference made based on this test’s scores. Further implications are discussed.
    Keywords: Constructs validity, Confirmatory factor analysis, Explanation inference, Grammatical knowledge
  • RÜdiger Grotjahn Page 20
    This article comments on theoretical misconceptions and misuses of statistics in Khodadady & Hashemi’s (2011) paper “Validity and C-Tests: The Role of Text Authenticity”. Firstly, it is pointed out that the pertinent C-Test literature is not adequately dealt with. Then, it is argued that the authors misconstrue the notion of the C-Test when they apply the term to a single (longer) C-Test text such as their AC-Test (Authentic C-Test). Subsequently, it is shown that there are serious flaws in the data analysis and interpretation. Here, the main focus is on local item dependence, which is not taken into account, and misconceptions with regard to Cronbach’s Alpha, an issue of relevance to a wider audience.
    Keywords: C, Test, Authenticity, Reliability, Cronbach's Alpha, Guttman's Lambda2, Local Stochastic Dependence, LID
  • Vahid Aryadoust Page 28
    Several studies have evaluated sentence structure and vocabulary (SSV) as a scoring criterion in assessing writing, but no consensus on its functionality has been reached. The present study presents evidence that this scoring criterion may not be appropriate in writing assessment. Scripts by 182 ESL students at two language centers were analyzed with the Rasch partial credit model. Although other scoring criteria functioned satisfactorily, SSV scores did not fit the Rasch model, and analysis of residuals showed SSV scoring on most test prompts loaded on a benign secondary dimension. The study proposes that a lexico-grammatical scoring criterion has potentially conflicting properties, and therefore recommends considering separate vocabulary and grammar criteria in writing assessment.
    Keywords: Lexico, grammatical scoring, Rasch model, Partial credit model (PCM), L2 writing, Writing models
  • Cynthia S. Wiseman Page 59
    This study compared the performance of a holistic and an analytic scoring rubric to assess ESL writing for placement and diagnostic purposes in a community college basic skills program. The study used Rasch many-faceted measurement to investigate the performance of both rubrics in scoring second language (L2) writing samples from a departmental final examination. Rasch analyses were used to determine whether therubrics successfully separated examinees along a continuum of L2 writing proficiency. The study also investigated whether each category in the two six-point rubrics were useful. Both scales appeared to be measuring a single latent trait of writing ability. Raters hardly used the lower category of the holistic rubric, suggesting that it might be collapsed to create a five-point scale. The six-point scale of the analytic rubric, on the other hand, separated examinees across a wide range of strata of L2 writing ability and might therefore be the better instrument in assessment for diagnostic and placement purposes.
    Keywords: L2 writing assessment, Analytic scoring, Holistic scoring, Rubrics, Rasch, MFRM