Contrasting Groups Analysis of TOEFL® iBT Test Cut Scores and the Common European Framework of Reference (CEFR) Proficiency Levels: Kernel Density Estimation of an English Learners’ Corpus
منبع:
International Journal of Language Testing, Volume ۱۱, Issue ۱, Winter and Spring ۲۰۲۱
88 - 102
حوزههای تخصصی:
Placing non-native speakers of English into appropriate classes involves mapping placement test scores onto proficiency levels based on predetermined cut scores. However, studies on how to set boundaries for different levels of proficiency have been lacking in the language testing literature. A top-down approach to standard setting in which a panel of experts set cut scores has dominated the typical standard setting procedure. A less utilized approach is to proceed bottom-up by clustering learners based on test scores. The purpose of this study was to fill this gap by examining Education Testing Services (ETS)’s mapping of TOEFL® iBT Test scores to the Common European Framework of Reference (CEFR) levels. The study examined TOEFL® iBT score data from ICNALE (International Corpus Network of Asian Learners of English) and conducted optimal Kernel Density Estimation to find peaks in the distribution of test scores. In addition to the number of peaks, the local minima of the resulting distribution were chosen as boundaries of cut scores for delineating different ability groups. This method of separating scores, also known as contrasting groups, finds clusters of test takers based on maximum differences in scores. The results showed that ETS’ guide for cut scores linking to CEFR levels was comparable to Kernel Density Estimation with some exceptions, namely two out of three cut scores were found to be similar. Implications are discussed in terms of test-centered versus examinee-centered method of standard setting and the need to consider the demographics of the examinee population in determining cut scores.