G-DINA

Cognitive diagnostic models (CDMs) have received much interest within the field of language testing over the last decade due to their great potential to provide diagnostic feedback to all stakeholders and ultimately improve language teaching and learning. A large number of studies have demonstrated the application of CDMs on advanced large-scale English proficiency exams, such as IELTS, TOEFL, MELAB, and ECPE. However, too little attention has been paid to the utility of CDMs on elementary and intermediate high-stakes English exams. The current study aims to diagnose the reading ability of test takers in the B1 Preliminary test, previously known as the Preliminary English Test (PET), using the generalized deterministic input, noisy, “and” gate (G-DINA; de la Torre, 2011) model. The G-DINA is a general and saturated model which allows attributes to combine in both compensatory and non-compensatory relationships and each item to select the best model. To achieve the purpose of the study, an initial Q-matrix based on the theory of reading comprehension and the consensus of content experts was constructed and validated. Item responses of 435 test takers to the reading comprehension section of the PET were analyzed using the “G-DINA” package in R. The results of attribute profiles suggested that lexico-grammatical knowledge is the most difficult attribute, and making an inference is the easiest one.

The purpose of this study was to compare the functioning of five restrictive CDMs, including DINA, DINO, A-CDM, LLM, and RRUM, against the G-DINA model to identify the best-fitting CDM which can better explain the interaction underlying the attributes of the reading comprehension section of an Iranian high-stakes language proficiency test. To achieve this aim, item responses of 1152 examinees to the items of the test were examined. The six CDMs were initially compared in terms of relative and absolute fit statistics at test-level to choose the best model. It was found that the G-DINA model outperformed compared to the restrictive models; thus, it was selected for the second phase of the study. Concerning the second purpose of the study, the G-DINA was used to identify strengths and weaknesses of the examinees. The results revealed that making an inference and vocabulary are the hardest attributes for examinees of the test, and understanding the specific information is the easiest attribute. Finally, the models were also compared at item-level. The presence of a combination of L2 reading attributes was found.