International Journal of Language Testing

Using Kane's interpretive argument model and Messick's validity argument approach, this study rigorously examined faculty and PhD candidate’s perspectives on PhD admission interviews in Iranian universities. We interviewed 10 professors and PhD interviewees which provided comprehensive insight into nuanced perspectives. We conducted rigorous content analysis to identify prevalent themes, forming a strong foundation for our analysis. This study emphasizes the vital requirement for standardized evaluation criteria, robust support systems, and an enhanced interview process to ensure fair and inclusive admission systems. Additionally, our development of guidelines based on Toulmin's reasoning model underscores the originality of our contribution and its potential to benefit stakeholders and the Ministry of Science, Research, and Technology (MSRT) in Iran. The findings highlighted the importance of standardized criteria, support, and a stronger interview process for fairness and inclusivity in selecting PhD candidates. Faculty stressed clear guidelines to remove subjectivity, while candidates voiced concerns about unclear expectations and proposed added support like mentoring and preparation programs. Based on Toulmin's reasoning model, the study crafted validity argument guidelines for this context. As a result, these proposed changes will impact stakeholders and the MSRT by enhancing the PhD candidate evaluation process and ensuring a fairness and inclusivity. This study provides valuable insights to improve PhD admission procedures at Iranian universities by integrating standardized criteria, enhancing support mechanisms, and fostering fairness in decision-making.

Despite being a popular topic in language testing, cognitive load has not received enough attention in vocabulary test items. The purpose of the current study was to scrutinize the cognitive load and vocabulary test items’ differences, examinees’ reaction times, and perceived difficulty. To this end, 150 students were selected using cluster/convenience-sampling, and took the Cambridge Placement Test (CPT) and Vocabulary Size Test (VST; Nation & Beglar, 2007). After uploading the vocabulary-size test’s items in PsychoPy software, there was a behavioral stage to measure students’ reaction times and correct responses. Out of these 150 high school students, a total of 60 (20 from each proficiency level of elementary/intermediate/advanced groups) were selected. In this quantitative study, all 60 students were interviewed to determine their perceived difficulty of the international VST items and their item’s difficulty-index. The data were analyzed quantitatively via simple regression and qualitatively through the examination of the students’ perceived difficulty. The results and interview findings revealed a significant connection between cognitive load/reaction time, difficulty estimate, and perceived difficulty at intermediate level. In contrast, at elementary and advanced levels, these variables could not predict the cognitive load. The findings can help to test, course, and syllabus designers by educating them on the significance of cognitive load theory so that they can base their exam designs on its premises and alleviate students' increased cognitive-workload.

This research paper delved into the critical issue of applying English as a Lingua Franca (ELF) assessment principles in local English language tests used for non-native English speakers in Iranian language institutes. A qualitative content analysis was made on 60 local tests, dissecting them into domains, dimensions, and rating rubrics to scrutinize their alignment with ELF assessment principles. The study unveiled that despite some alignment with ELF assessment principles, key aspects like local communicative context, intercultural competence, and linguistic diversity are often overlooked. In particular, writing and reading tests failed to fully reflect these principles, and listening and speaking assessments showed biases towards native English varieties. The study provides crucial insights for test developers to foster a more nuanced and accurate assessment of non-native English speakers' abilities. Moreover, it highlights the need to embed ELF principles into test construction, argues for broader assessment scopes and a focus on locally relevant tasks, and contributes to more equitable and contextually relevant English language proficiency tests by emphasizing linguistic diversity in assessment frameworks.

Construct-irrelevant variance is considered as a major threat to validity which indicates the existence of additional unrelated variables that distort the meaning of test scores and cause the test to be biased. Differential item functioning (DIF) analysis is an important technique in examining the validity and fairness of educational tests. Concerning the importance of test fairness in large-scale exams, this study aimed to (1) detect gender DIF in the reading comprehension section of the B2 First exam using the Rasch model and Mantel-Haenszel method, and (2) investigate the comparability of results from the two DIF detection techniques. To this end, the reading section of the B2 First exam was administered to 207 undergraduate students of English as a foreign language (EFL). After checking the fit of the data to the Rasch model, the results of the Rasch model-based DIF analysis showed the presence of two items indicating DIF, whereas the results of Mantel-Haenszel showed that there were three gender-DIF items.

Communication skill is considered to be one of the most demanding skills of the day, and proficiency in listening and speaking skills paves the way for being communicative. To develop one’s listening and speaking skills, there are many ways and factors including the use of audiovisual materials in an English as a foreign language (EFL) classroom. This study analyzed the effectiveness of audiovisual materials in developing tertiary level students' English listening and speaking skills. To do so, experiments followed by formative assessment were carried out on the first-year undergraduate students of the Department of English, Jashore University of Science and Technology, Bangladesh. To triangulate the experiment results and ensure trustworthiness, a student-questionnaire survey was also conducted. The overall findings revealed that the use of audiovisual materials has a major impact on the development of listening and speaking skills of EFL students. The findings of this research are supposed to help education policymakers, education administrators, teachers, and students to adopt better policies and decisions to make teaching and learning English more effective and fruitful at different stages in second or foreign language contexts.

In tandem with communicative approaches to language teaching, there is a growing understanding in SLA that assessment needs to be integrated into language learning. This contention has led to the development of the Learning-Oriented Assessment (LOA) approach for assessing language skills and aspects in language classrooms. To our knowledge, interventionist studies to improve EFL teachers' LOA-based assessment practices are in their infancy. The present study examined Iranian EFL teachers' perspectives on LOA in language teaching. To this end, the researchers selected 44 EFL teachers in four language institutes in Urmia (Iran) and provided them with tailor-made LOA-based training in 12 sessions for one month. The teachers were informed that LOA procedures are to be adopted and used in their classes. During the following semester, the researchers observed the participants' classes periodically and provided them with comments on their LOA procedures. Following the course, interviews were held with the participants to probe into their perspectives on LOA. Drawing upon thematic analysis, the researchers analyzed the recordings to develop a model for the participants' perspectives on LOA procedures. Development of teacher noticing skills in LOA procedure, its beneficial impact on language learning vis-à-vis traditional testing procedures, implementation challenges, widespread use at all proficiency levels, and the need to develop user-friendly LOA-compatible software and applications constituted the major themes based on the interview data. The findings demonstrate that the participants harbored favorable views on LOA and regarded it to be more efficacious than traditional testing procedures.

Cloze tests are commonly used in language testing as a quick measure of overall language ability or reading comprehension. A problem for the analysis of cloze tests with item response theory models is that cloze test items are locally dependent. This leads to the violation of the conditional or local independence assumption of IRT models. In this study, a new modeling strategy is suggested to circumvent the problem of local item dependence in cloze tests. This strategy involves identifying locally dependent items in the first step and combining them into polytomous items in the second step. Finally, partial credit model is applied to the combination of dichotomous and polytomous items. Our findings showed that the new strategy results in a better model-data fit than the dichotomous model where dependence is ignored but with a lower reliability. Results also indicated that the person and item parameters from the two models highly correlate. The findings are discussed in light of the literature on managing local dependence in educational tests.

Becoming one of six official United Nations languages, English proficiency has become a prominent requirement for students and professionals entering international education and/ or careers, as assessed through standardized English proficiency tests. English learners conducted varied procedures to pursue the minimum test score criteria, including joining an English test preparation program. This study investigates the effect of English test preparation as an intervention on EFL learners' English language proficiency performance by reviewing the effect sizes found by previous studies using meta-analysis of pre-post contrasts. A total of 20 selected studies, according to the criteria determined by researchers between 2018-2021 and conducted in various countries, were collected from Google Scholar sources and included in the research. English language proficiency performance, as indicated by their test scores, is independent of the language skills tested with a p-value of less than 0.05 (p-value = 0.00) and an average weighted effect (M) that appears to be 0.859. This research implies that adequate test preparation is essential for English language learners who wish to achieve the desired level of proficiency. Apart from the ability to write, speak, and read, the psychological aspects of students are also important to prepare before they take the intended test

While the potential of Dynamic Assessment (DA) and its variants (Computerized Dynamic Assessment (CDA) and Group Dynamic Assessment (GDA)) for EFL classrooms has been recognized, there is a lack of research on its practical implementation compared to the well-established field of Summative Assessment (SA). Thus, the objective of this qualitative study was to investigate the evolving perspectives of EFL teachers concerning the integration of DA and SA within their classrooms. To achieve this, 50 EFL teachers in Iran were recruited through convenience sampling to complete an online open-ended questionnaire. The primary purpose was to explore their familiarity with, perceptions of practicality for, and preferences regarding DA and SA. Additionally, a sub-group of volunteer participants was requested to provide narratives detailing their real-world classroom experiences using DA and SA. Content and thematic analysis of the responses revealed that the majority of participants were familiar with DA, with the most commonly employed type being GDA. While DA was predominantly viewed as a form of feedback, SA was still seen as a more formal means of classroom assessment. Consequently, it is highly recommended that EFL teachers exploit the advantages of both assessment approaches in order to ensure more equitable decisions concerning students' abilities.

In recent years, dynamic assessment and positive psychology have attracted the attention of many researchers. This phenomenological study explores Iranian intermediate English as a Foreign Language (EFL) learners' perception of academic buoyancy, reflective thinking, and academic resilience in response to dynamic assessment. Data were gathered through narrative inquiry, observation, and focus group discussion involving 18 intermediate EFL learners at a language institute in South Iran. Member checking, peer debriefing, and audit trail were used to ensure the credibility and dependability of the instruments. Thematic analysis of the qualitative data revealed that dynamic assessment positively influenced learners' academic buoyancy by providing tailored scaffolding and support, fostering resilience in the face of academic challenges, and enhancing reflective thinking abilities. These findings suggest that integrating dynamic assessment techniques into language teaching practices contributes to students' adaptive coping mechanisms and ability to navigate academic setbacks, enhancing their academic success and overall well-being. The study underscores the importance of incorporating dynamic assessment approaches to cultivate resilient and empowered learners within EFL settings. This study contributes to understanding dynamic assessment's role in fostering academic resilience and reflective thinking in language learning contexts. The implications of the study are discussed.

This study aims to assess the performance of university students through the C-Test and to analyze the extent to which this test is valid in measuring language ability. A standardized C-Test has been created with four brief passages, each containing 20 gaps. The length of each passage varied from 95 to 109 words. Throughout each passage, only the first and last sentences were not changed. The test was taken by 100 students; 39 were male and 61 were female at Al-Nisour University/Department of English in Baghdad, Iraq. The sample consists of two groups. Both groups come from the same school and would receive similar educational input in both cases based on their grade level. The validity and reliability of the C-Test were investigated using various techniques. The study analyzed the performance of Stage 4 and Stage 3 students on the Common Language Proficiency Test in Iraq. The results showed that the test discriminates well between high-ability and low-ability examinees, with no significant difference between the two groups. The Rasch model separation reliability was relatively high, and the data were one-dimensional. The students faced difficulties in guessing the most appropriate words due to their limited English proficiency. The results suggest that developing and implementing this test could significantly improve students' academic achievements in basic foreign language classes in Iraq.