مطالب

Measurement has been ubiquitous in all areas of education for at least a century. Various methods have been suggested to examine the fairness of education tests especially in high-stakes contexts. The present study has adopted the newly proposed ecological approach to differential item functioning (DIF) to investigate the fairness of the Iranian nationwide university entrance exam. To this end, the actual data from an administration of the test were obtained and analyzed through both traditional logistic regression and latent class analysis (LCA) techniques. The initial DIF analysis through logistic regression revealed that 19 items (out of 70) showed either uniform or non-uniform DIF. Further examination of the sample through LCA showed that the sample is not homogeneous. LCA class enumeration revealed that three classes can be identified in the sample. DIF analysis for separate latent classes showed that three serious differences in the number of DIF items identified in each latent class ranging from zero items in latent class 3 to 43 items in latent class 2. The inclusion of the covariates in the model also showed that latent class membership could be significantly predicted from high school GPA, field of study, and acceptance quota. It is argued that the fairness of the test might be under question. The implications of the findings for the validity of the test are discussed in detail.

Since its inception in 1949, over 1,500 studies have investigated the validity of the GRE General Test to predict its performance criteria in higher education (Klieger, Bridgeman, Tannenbaum, & Cline, 2016). The present review paper sought to examine the predictive validity of the GRE General Test. Factors affecting the predictive validity (e.g., range restriction, compensatory selection, criterion unreliability, substantive and artifactual moderators, bias in testing, coaching effects, socioeconomic status (SES), gender, and a host of other intervening factors such as motivation, communication skills, etc.) have been discussed. A brief overview of GRE revised General Test format is also presented. After an account of the related review of the literature, a critical commentary on the predictive validity of the GRE General Test has been discussed with an emphasis on the role of criterion unreliability and SES factor effects. Keywords: Predictive validity, Graduate Record Examination (GRE), criterion unreliability, range restriction, compensatory selection

Large-scale standardized ESL tests such as the International English Language Testing System (IELTS) are widely used around the world to measure language proficiency of test takers and make different decisions based on their scores. Reading comprehension is an integral part of such tests which requires test takers to read passages and answer a set of questions. Although IELTS is a popular standardized test and is used for making critical decisions about test takers, very few attempts have been made to explore the validity of the exam and especially the reading part of the General Training Module. With this in mind, the purpose of the present study was to use a non-parametric item response theory model, called Mokken Scale Analysis (MSA), to examine the validity of the reading part of the General Training module of IELTS. To this end, item responses of 352 test takers to the reading comprehension test were analyzed. The results of item scalability, total scalability, and item-pair scalability showed that the reading part is a weak unidimensional scale. Using Monotone Homogeneity Model (MHM), monotonicity results also indicated that there are some items which violate the monotonicity assumption, although their values are insignificant. The analysis of unidimensionality using the AISP revealed that there are two scales and four unscalable items in the reading part. Therefore, Mokken scale analysis did not support the unidimensional structure of the reading part of the General Training module of IELTS.

This study examined the development of text complexity for the past 25 years of reading comprehension passages in the National Matriculation English Test (NMET) in China. Text complexity of 206 reading passages at lexical, syntactic, and discourse levels has been measured longitudinally and compared across the years. The natural language processing tools used in the study included TAALES, TAALED, TAASSC, and TAACO. To compare the differences across the years at various levels of text complexity, ANOVA and MANOVA tests were conducted. The results suggested that lexical level text complexity revealed the most evident changes throughout the years, lexical sophistication, density, and diversity levels of the most recent years of reading passages have increased remarkably compared to the early years. The syntactic level text complexity indicated a moderate elevation toward the recent years of reading passages. For the discourse level text complexity, regarding cohesion, insignificant fluctuation occurred throughout the years and the general trend was not necessarily increasing. Combined, the results indicated that text complexity of the reading comprehension passages in the NMET over the past 25 years had been steadily increasing by including more low frequency and academic vocabulary, diversifying vocabulary in the passages, and complicating sentence and grammatical structures. The results were further examined against the general curriculum standards and guidelines to analyze whether the changes were reflected in the policies. It showed that the exams required a much larger vocabulary size than the number indicated in the guidelines, suggestions for test designers and pedagogical practices were provided accordingly.