سید محمد علوی

Assessment of writing skill is generally believed to be judged by a rater subjectively and qualitatively or by using analytic scoring rubrics which can potentially result in somehow not very reliable assessment. It seems that an evaluation of writing based on a model can result in a valid and reliable writing assessment. To achieve such an objective, this study firstly aimed to develop an assessment model based on Activity Theory (AT), i. e., Activity Theory-Based Assessment Model (ATBAM), and then to employ it in the assessment of writing performances of Iranian language learners in a private language college. And finally, to achieve the concurrent validity of ATBAM, its results were compared with those of a traditional approach. Three groups of participants took part in this study: a group of upper intermediate English learners (N=29) who submitted one writing sample per week in four successive weeks, teachers (N=6) who provided learners with feedback and assigned holistic scores and course supervisors (N=2) who reassessed the writing samples on the basis of an analytic rubric of writing assessment. The results showed that using ATBAM in writing assessment results in an exploration of not only learners’ but also teachers’ engagement in the development of learners’ writing ability. The role of teachers’ feedback and teachers’ and learners’ social interaction in the development of learners’ writing ability could potentially provide comprehensive, fair, reliable, and valid scores in this model.

When constructing a test, an initial decision is choosing an appropriate item response format which can be classified as selected or constructed. In large-scale tests where time and finance are of concern, the use of response chosen known as multiple-choice items is quite widespread. This study aimed at investigating the impact of response format on the performance of structure tests. Concurrent common item equating design was used to compare multiple-choice items with their constructed response stem-equivalent in a test of grammar. The Rasch model was employed to compare item difficulties, fit statistics, ability estimates and reliabilities of the two tests. Two independent sample t-tests were also conducted to investigate whether the differences among the item difficulty estimates and ability estimates of the two tests were statistically significant.  A statistically significant difference was observed in item difficulties. However, no significant difference was detected between the ability estimates, fit statistics, and reliabilities of the two tests.

Although Diagnostic Classification Models (DCMs) were introduced to education system decades ago, it seems that these models were not employed for the original aims upon which they had been designed. Using DCMs has been mostly common in analyzing large-scale non-diagnostic tests and these models have been rarely used in developing Cognitive Diagnostic Assessment (CDA) from scratch. Despite the prevalence of retrofitting CDA studies, true applications of CDA are believed to be rare since, firstly, a coherent framework to conduct such studies had not been available and, secondly, researchers were not able to analyze various DCMs according to the same model fit indices and criteria. This paper presents a summary of different types of DCMs and reviews true and retrofitting CDA studies. Having examined the limitations of previous CDA studies, the present study argues for the implication and application of Ravand and Baghaei’s (2019) framework to conduct true CDA studies. This framework is of importance since not only does it fit into prominent frameworks in education assessment such as Cognitive Design System and Assessment Triangle, but also it can provide test-developers with practical steps in conducting valid cognitive diagnostic tests.

Formative writing assessment can help writing instructors to explore weaknesses and strengths of language learners’ writing performances. The current research aimed to explore firstly writing attributes and secondly examine their reciprocal contribution to one another. To achieve such an objective, the participants (N=200) were asked to write about two different topics. One writing sample before treatment was considered as the pre-test and the other after the treatment was considered as the post-test writing sample. Having scrutinized the pre-writing samples, five raters extracted the writing attributes which appeared in pre-test and post-test writing samples. Results indicate that there is a statistically significant difference among the participants’ performances in terms of using writing attributes. The results can be advantageous for both instructors and syllabus designers to provide pedagogical materials which identify particular frailties and notify them about the more troublesome points to concentrate on in the classroom so as to arrange effective education.

Computerized Dynamic Assessment (CDA), encouraged by Brown and colleagues’ graduated prompt approach, is grounded in Vygotsky’s Socio-Cultural Theory (SCT) of mind and its concept of the zone of proximal development (ZPD). It emerged to respond to the challenge of implementing DA in large classes and to meet the psychometric properties of assessment. To this end, the present study attempted to design a unique computerized dynamic assessment tool to diagnose learners’ development of pragmatic competence, specifically their knowledge of the speech acts of apology and request. To conduct the research, a number of 60 BSc students of engineering, aged 18-24, participated in the study. They had different proficiency levels, including pre-intermediate, intermediate and upper-intermediate levels. In the course of CDA, they were provided with 30 multiple choice discourse completion tests of apology and request and they were required to choose what they would say in that specific situation. The participants received pre-established meditational hints for each of the unacceptable responses, which were arranged from the most implicit to the most explicit. Finally, to diagnose learners’ development, their test performance, including their actual score, mediated score and learning potential score (LPS), was instantly displayed. Paired samples t-test showed development in learners’ mediated score. The results of the univariate analysis of variance showed that there is no interaction between mediation and proficiency level. Teachers can use this supplementary dynamic assessment tool to diagnose learners’ development of pragmatic competence.

The current study attempted qualitatively to explore and compare the qualities that native and Iranian English teachers (with and without related educational backgrounds) attend to while rating their students' oral productions in the classroom context. In doing so, the perceptions of 19 native English teachers (9 graduates in TEFL and 10 graduates in other majors) along with 18 Iranian English teachers (10 graduates in TEFL and 8 graduates in other majors) were sought through semi-structured interviews. The data were collected after the outbreak of Coronavirus (COVID-19) disease in 2020, which gave the researchers no choice but to look for haphazard cases with specific features in social networks. The recorded interviews were analyzed attentively through content analysis. The findings indicated that although all native and non-native respondents focused intensively on the structural features of language in general while rating oral interviews, they had notably different views regarding some sub-features within each category. Further results showed that the native and non-native TEFL-graduate teachers, unlike their peers with unrelated educational backgrounds, also gave credits to several message-based and pragmatic aspects of oral production. The findings have practical implications for researchers, pre-service and in-service teachers, and teacher educators.

Writing is thought as the most complicated skill in second language acquisition; therefore, L2 researchers have always been in pursuit of discovering an effective approach to improve it. One of the most debated ways is feedback which has a key role in improving the quality of writing. Much of the previous research on feedback has focused on analyzing different types of feedback and their effect on the learners’ writings and few studies have examined the effectiveness of computer feedback. Therefore, the present study was conducted to 1) determine what aspects of students’ writings receive computer feedback, 2) examine the difference in the effect of computer-generatedfeedback (CBF) and Teacher-based feedback (TBF) on improving the students’ writing quality and 3) compare the differences in Depth of Processing (DOP) in processing computer and teacher feedback. The results indicated that content, style and organization of their essays received feedback from the teacher and the computer. Teacher feedback was more effective in terms of its impact on improving the quality of the writing of the students than computer-generatedfeedback and it resulted in deeper processing of lexical items, whereas computer-generated feedback invoked medium processing on grammar.