Application of Confirmatory Factor Analysis in Construct Validity Investigation: The Case of the Grammar Sub-Test of the CEP Placement Exam

An important assumption in language testing is that test items or observable variables tap the underlying latent traits hypothesized in the theoretical model or constructs governing the design of the testing instrument (e.g., Shin, 2005). Accordingly, the present study sought to investigate the extent to which scores from the grammar sub-test of the Columbia University Community English Program (CEP) placement test could be interpreted as indicators of test takers’ grammatical knowledge. The authors adopted Pupura’s (2004) theoretical model of grammatical knowledge, which hypothesizes that grammatical knowledge consists of two underlying traits of form and meaning. To this end, the authors conducted a confirmatory factor analysis (CFA) to investigate whether there is a match between the CEP grammar test data (n= 144) and the theoretical model as hypothesized. Since the test items were not discrete point but were nested within one of four tasks (each with their own theme), by endorsing the interactionist view of construct definition, effects of these four themes (context) on individual items were also investigated. A multitrait-multimethod matrix (MTMM) model achieved the best possible model fit based on substantive and parsimony considerations. It included two underlying traits of grammatical form and meaning and four method (context) factors, and confirmed that the CEP test examined the grammatical knowledge and included the effect of context as a part of its construct. These findings support the interpretive argument presented for the construct validly of the CEP grammar test, and the appropriateness of the explanation inference made based on this test’s scores. Further implications are discussed.