علیرضا احمدی

Score decision-making is largely an undocumented process in performance assessment. To conduct a more in-depth cognitive study in scoring, one must ask if these underlying processes can be identified efficiently and objectively. To this end, the present study attempted to shed some light on how Iranian teachers as untrained raters rate speech samples of learners and how their cognition functions in the decision-making process in terms of the scores they assign. A series of monologues were obtained from a group of language learners; afterward, English language teachers were asked to rate them. The raters were asked both to assign a score and provide comments regarding why they assigned a specific score. Having rated the samples, the raters were individually interviewed. The results of the recorded interviews and the comments they had provided on scores were subjected to qualitative analysis like coding and extracting both idiosyncratic and shared features of the raters’ cognition. The results revealed that some of the factors attended to by the raters were both linguistic and relevant to speaking proficiency construct like fluency, accuracy, and complexity. Some other factors influencing the raters while rating were non-linguistic and not directly related to speaking proficiency construct like the tone of voice, personality feature of the testee, etc. It seemed that the untrained raters did not have a clear definition of oral proficiency construct. The implications of the study for rater training programs have been discussed.

The score reliability of language performance tests has attracted increasing interest. Classical Test Theory cannot examine multiple sources of measurement error. Generalizability theory extends Classical Test Theory to provide a practical framework to identify and estimate multiple factors contributing to the total variance of measurement. Generalizability theory by using analysis of variance divides variances into their corresponding sources, and finds their interactions. This study used generalizability theory as a theoretical framework to investigate the effect of raters’ gender on the assessment of EFL students’ writing. Thirty Iranian university students participated in the study. They were asked to write on an independent task and an integrated task. The essays were holistically scored by 14 raters. A rater training session was held prior to scoring the writing samples. The data were analyzed using GENOVA software program. The results indicated that the male raters’ scores were as reliable as those of the female raters for both writing tasks. Large rater variance component revealed the low score generalizability in case of using one rater. The implications of the results in the educational assessment are elaborated.

The purpose of the study was to investigate the effect of mediation on the development of a novice teacher and in turn the effect of transformation of the teacher on the behaviors and emotions of the learners using Vygotskian sociocultural view of learning. For this purpose, a novice teacher teaching the general English course at an Iranian university was selected. To develop an understanding of the instructional setting and to diagnose her weaknesses and strengths, she was observed by a teacher educator during the first five sessions. Different instruments and tools like the teacher and students’ narratives, stimulated recall of voice-recorded actual teaching, and observation were also used during the study to trace the teacher’s cognitive and emotional development. As most of her practices displayed over-reliance on translation and deductive teaching of grammar, and her oral narration showed instances of cognitive/emotional dissonance, strategic mediations, both implicit and explicit, based on her needs were offered after the fifth session in a face-to-face interaction to help her develop gradually. Analysis of the results showed the gradual development of her and the positive attitudes and active engagement of the learners during she was being mediated. Implications of the findings are presented and discussed.

This study applied multi-faceted Rasch measurement to investigate the halo effect in the performance-based assessment of writing across rubric criteria. Five raters who had received specialized training applied a four-criteria rating rubric to analytically rate writing scripts on two argumentative topics. Facets, a Rasch computer program, was utilized to pinpoint the halo effect by analyzing rater and rubric criteria interactions. After examining the appropriateness of the rubric in terms of functionality, the results showed that except for one rater, the raters did not exhibit any sign of the halo effect across rubric criteria. Generally, the severity hierarchies for raters and difficulty levels for rubric criteria suggested that raters’ training and their perceptions of the importance of criteria were associated with their manifestation of the halo effect. Pedagogically, through a detailed facet analysis of interactions between raters and rubric criteria, rater trainers may better realize how to design effective training programs to minimize raters’ variance including the halo effect and improve the overall objectivity of human rating.

Background: Sports facilities are one of the most commonly used services in the city with a significant role in improving the physical and mental health condition of citizens thus a proper procedure is required to locate and distribute them. Aim : The purpose of this study is to design a comprehensive model for the optimal location of sports facilities. Materials and Methods: The research method is descriptive-analytic based on information gathering and is applied research based on objectives. The opinions of 20 experts have been used to design the model using the Delphi method, and weighting the effective criteria in the sports facilities location. The weight of each criterion has been obtained as population density (0.47), access (0.31), development potential (0.14), and adjacency (0.08); furthermore, the Kendall coefficient of concordance (0.74) in the third step of the Delphi method shows the strong agreement between the experts, regarding the proposed model. The proposed model consisted of six steps: 1. Aim; 2. Verification of the functional area of the existing sport facilities and specific restrictions of the area; 3. Introducing and weighting the important criteria in the sports facilities location; 4. Identifying the most suitable locations for constructing the sports facilities; 5. Evaluating the needs of users; 6. Selecting the best spaces and prioritizing them. Results: The results of the model showed that the most important criteria for locating sport facilities are population density, access, development potential, and adjacency. Additionally, it was indicated that the agreement between the experts increased over time. Conclusion: According to the proposed model, it is possible to identify the points that are suitable for constructing the new sports facilities.