سید محمد علوی

Assessment of writing skill is generally believed to be judged by a rater subjectively and qualitatively or by using analytic scoring rubrics which can potentially result in somehow not very reliable assessment. It seems that an evaluation of writing based on a model can result in a valid and reliable writing assessment. To achieve such an objective, this study firstly aimed to develop an assessment model based on Activity Theory (AT), i. e., Activity Theory-Based Assessment Model (ATBAM), and then to employ it in the assessment of writing performances of Iranian language learners in a private language college. And finally, to achieve the concurrent validity of ATBAM, its results were compared with those of a traditional approach. Three groups of participants took part in this study: a group of upper intermediate English learners (N=29) who submitted one writing sample per week in four successive weeks, teachers (N=6) who provided learners with feedback and assigned holistic scores and course supervisors (N=2) who reassessed the writing samples on the basis of an analytic rubric of writing assessment. The results showed that using ATBAM in writing assessment results in an exploration of not only learners’ but also teachers’ engagement in the development of learners’ writing ability. The role of teachers’ feedback and teachers’ and learners’ social interaction in the development of learners’ writing ability could potentially provide comprehensive, fair, reliable, and valid scores in this model.

Formative writing assessment can help writing instructors to explore weaknesses and strengths of language learners’ writing performances. The current research aimed to explore firstly writing attributes and secondly examine their reciprocal contribution to one another. To achieve such an objective, the participants (N=200) were asked to write about two different topics. One writing sample before treatment was considered as the pre-test and the other after the treatment was considered as the post-test writing sample. Having scrutinized the pre-writing samples, five raters extracted the writing attributes which appeared in pre-test and post-test writing samples. Results indicate that there is a statistically significant difference among the participants’ performances in terms of using writing attributes. The results can be advantageous for both instructors and syllabus designers to provide pedagogical materials which identify particular frailties and notify them about the more troublesome points to concentrate on in the classroom so as to arrange effective education.

Although Diagnostic Classification Models (DCMs) were introduced to education system decades ago, it seems that these models were not employed for the original aims upon which they had been designed. Using DCMs has been mostly common in analyzing large-scale non-diagnostic tests and these models have been rarely used in developing Cognitive Diagnostic Assessment (CDA) from scratch. Despite the prevalence of retrofitting CDA studies, true applications of CDA are believed to be rare since, firstly, a coherent framework to conduct such studies had not been available and, secondly, researchers were not able to analyze various DCMs according to the same model fit indices and criteria. This paper presents a summary of different types of DCMs and reviews true and retrofitting CDA studies. Having examined the limitations of previous CDA studies, the present study argues for the implication and application of Ravand and Baghaei’s (2019) framework to conduct true CDA studies. This framework is of importance since not only does it fit into prominent frameworks in education assessment such as Cognitive Design System and Assessment Triangle, but also it can provide test-developers with practical steps in conducting valid cognitive diagnostic tests.

The current study attempted qualitatively to explore and compare the qualities that native and Iranian English teachers (with and without related educational backgrounds) attend to while rating their students' oral productions in the classroom context. In doing so, the perceptions of 19 native English teachers (9 graduates in TEFL and 10 graduates in other majors) along with 18 Iranian English teachers (10 graduates in TEFL and 8 graduates in other majors) were sought through semi-structured interviews. The data were collected after the outbreak of Coronavirus (COVID-19) disease in 2020, which gave the researchers no choice but to look for haphazard cases with specific features in social networks. The recorded interviews were analyzed attentively through content analysis. The findings indicated that although all native and non-native respondents focused intensively on the structural features of language in general while rating oral interviews, they had notably different views regarding some sub-features within each category. Further results showed that the native and non-native TEFL-graduate teachers, unlike their peers with unrelated educational backgrounds, also gave credits to several message-based and pragmatic aspects of oral production. The findings have practical implications for researchers, pre-service and in-service teachers, and teacher educators.

When constructing a test, an initial decision is choosing an appropriate item response format which can be classified as selected or constructed. In large-scale tests where time and finance are of concern, the use of response chosen known as multiple-choice items is quite widespread. This study aimed at investigating the impact of response format on the performance of structure tests. Concurrent common item equating design was used to compare multiple-choice items with their constructed response stem-equivalent in a test of grammar. The Rasch model was employed to compare item difficulties, fit statistics, ability estimates and reliabilities of the two tests. Two independent sample t-tests were also conducted to investigate whether the differences among the item difficulty estimates and ability estimates of the two tests were statistically significant.  A statistically significant difference was observed in item difficulties. However, no significant difference was detected between the ability estimates, fit statistics, and reliabilities of the two tests.

Computerized Dynamic Assessment (CDA), encouraged by Brown and colleagues’ graduated prompt approach, is grounded in Vygotsky’s Socio-Cultural Theory (SCT) of mind and its concept of the zone of proximal development (ZPD). It emerged to respond to the challenge of implementing DA in large classes and to meet the psychometric properties of assessment. To this end, the present study attempted to design a unique computerized dynamic assessment tool to diagnose learners’ development of pragmatic competence, specifically their knowledge of the speech acts of apology and request. To conduct the research, a number of 60 BSc students of engineering, aged 18-24, participated in the study. They had different proficiency levels, including pre-intermediate, intermediate and upper-intermediate levels. In the course of CDA, they were provided with 30 multiple choice discourse completion tests of apology and request and they were required to choose what they would say in that specific situation. The participants received pre-established meditational hints for each of the unacceptable responses, which were arranged from the most implicit to the most explicit. Finally, to diagnose learners’ development, their test performance, including their actual score, mediated score and learning potential score (LPS), was instantly displayed. Paired samples t-test showed development in learners’ mediated score. The results of the univariate analysis of variance showed that there is no interaction between mediation and proficiency level. Teachers can use this supplementary dynamic assessment tool to diagnose learners’ development of pragmatic competence.

Writing is thought as the most complicated skill in second language acquisition; therefore, L2 researchers have always been in pursuit of discovering an effective approach to improve it. One of the most debated ways is feedback which has a key role in improving the quality of writing. Much of the previous research on feedback has focused on analyzing different types of feedback and their effect on the learners’ writings and few studies have examined the effectiveness of computer feedback. Therefore, the present study was conducted to 1) determine what aspects of students’ writings receive computer feedback, 2) examine the difference in the effect of computer-generatedfeedback (CBF) and Teacher-based feedback (TBF) on improving the students’ writing quality and 3) compare the differences in Depth of Processing (DOP) in processing computer and teacher feedback. The results indicated that content, style and organization of their essays received feedback from the teacher and the computer. Teacher feedback was more effective in terms of its impact on improving the quality of the writing of the students than computer-generatedfeedback and it resulted in deeper processing of lexical items, whereas computer-generated feedback invoked medium processing on grammar.   

Sandwich and cake formats of interventionist Dynamic Assessment suffer from two problems: both are time-demanding, and the intervention presented to the learners varies from one learner to another.  To suggest a solution to the above problems in teaching and assessing reading strategies, the present researchers have introduced ‘sauce’-format mediation through which intervention is provided for the learners within ‘mediation boxes’ along the test. Thirty Iranian intermediate EFL learners participated in the study. They were first given a non-mediated reading pretest, and then they took the same test along with a ‘mediation box’ following each question to observe the probable effect of ‘sauce’-format mediation.  To measure the transcendence of the given mediation, the participants answered a non-mediated delayed posttest at the end.  Comparing the means of the mediated posttest and non-mediated pretest on the one hand and those of the non-mediated delayed posttest and the pretest on the other revealed that the learners benefitted significantly well from the ‘sauce’-format mediation and could transfer their learning to a novel context. Findings of this study are consistent with the previous research on DA that intervention supports learner development.

Learning-oriented assessment (LOA) seeks to support instruction by providing information about students ’ current learning conditions and their ultimate performance. This study scrutinizes the application of LOA in English as a Foreign Language (EFL) writing classes within Iranian language institutes. In addition, the current study examined the participants’ writing traits using the researchers-developed writing traits questionnaire during the implementation of LOA. Sixty upper-intermediate students (male & female) in two intact classes at a language institute in Tabriz, Iran, were taken as the participants. One of these classes was randomly picked as the control and the other one as the experimental group. During the treatment, learners of the experimental group carried out the relevant writing tasks based on the principles of LOA. On the other hand, the control group adopted the teacher-centered approach. 120 IELTS writing task 2 samples were collected and scored by using a scoring rubric (Wang & Liao, 2008) before and after the LOA treatment. To realize the differences in the mean scores of students in the control group and the experimental group, an independent sample t-test was used. The findings showed that the students of the experimental group benefited vastly from multiple drafting, self, and peer-evaluation, sharing learning goals with teachers and each other, and using feedforward techniques instead of feedback. The results might provide both theory and pedagogy implications for syllabus designers and language teachers with valuable guidelines on improving writing assessment in IELTS writing courses.

Measurement has been ubiquitous in all areas of education for at least a century. Various methods have been suggested to examine the fairness of education tests especially in high-stakes contexts. The present study has adopted the newly proposed ecological approach to differential item functioning (DIF) to investigate the fairness of the Iranian nationwide university entrance exam. To this end, the actual data from an administration of the test were obtained and analyzed through both traditional logistic regression and latent class analysis (LCA) techniques. The initial DIF analysis through logistic regression revealed that 19 items (out of 70) showed either uniform or non-uniform DIF. Further examination of the sample through LCA showed that the sample is not homogeneous. LCA class enumeration revealed that three classes can be identified in the sample. DIF analysis for separate latent classes showed that three serious differences in the number of DIF items identified in each latent class ranging from zero items in latent class 3 to 43 items in latent class 2. The inclusion of the covariates in the model also showed that latent class membership could be significantly predicted from high school GPA, field of study, and acceptance quota. It is argued that the fairness of the test might be under question. The implications of the findings for the validity of the test are discussed in detail.