partial credit model

A C-Test is a gap-filling test for measuring language competence in the first and second language. C-Tests are usually analyzed with polytomous Rasch models by considering each passage as a super-item or testlet. This strategy helps overcome the local dependence inherent in C-Test gaps. However, there is little research on the best polytomous Rasch model for C-Tests. In this study, the Rating Scale Model (RSM) and the Partial Credit Model (PCM) for analyzing C-Tests were compared. To this end, a C-Test composed of six passages with both RSM and PCM was analyzed. The models were compared in terms of overall fit, individual item fit, dimensionality, test targeting, and reliability. Findings showed that, although the PCM has a better overall fit compared to the RSM, both models produce similar test statistics. In light of the findings of the study, the choice of the best Rasch model for C-Tests will be discussed.

Dictation is a traditional technique for both teaching and testing overall language ability and listening comprehension. In a dictation, a passage is read aloud by the teacher and examinees write down what they hear. Due to the peculiar form of dictations, psychometric analysis of dictations is challenging. In a dictation, there is no clear boundary between the items and every word in the text is potentially an item. This makes the analysis of dictations with classical and modern test theories rather difficult. In this study, we suggest a procedure to make dictations analyzable with psychometric models. Our strategy entailed using several independent short passages instead of a single long passage. The number of mistakes in each passage was counted and entered into the analysis. Rasch model analysis was then applied to the passage scores (mistakes). Our findings showed that dictations fit the Rasch model very well and it is possible to measure examinees’ ability on an interval scale using dictations.

Cloze tests are commonly used in language testing as a quick measure of overall language ability or reading comprehension. A problem for the analysis of cloze tests with item response theory models is that cloze test items are locally dependent. This leads to the violation of the conditional or local independence assumption of IRT models. In this study, a new modeling strategy is suggested to circumvent the problem of local item dependence in cloze tests. This strategy involves identifying locally dependent items in the first step and combining them into polytomous items in the second step. Finally, partial credit model is applied to the combination of dichotomous and polytomous items. Our findings showed that the new strategy results in a better model-data fit than the dichotomous model where dependence is ignored but with a lower reliability. Results also indicated that the person and item parameters from the two models highly correlate. The findings are discussed in light of the literature on managing local dependence in educational tests.