Considering the body of research on the intersection of psychology and assessment that suggests assessment is not only under the influence of the model against which a translation is assessed, the present study aimed to investigate the role of raters’ personality traits in their assessment. It was also intended to see if the application of an agreed-upon unified assessment tool could change the way evaluations are made, and if this possible change is the same among raters with different personality traits. To this end, seven raters were asked to score an English text translated by 23 students twice with an interval of at least two weeks. In the first scoring, they were asked to assess the way they usually did with no specific guidelines. However, the second assessment was done through the application of an agreed-upon model. Moreover, the raters were scored on the personality traits of agreeableness and conscientiousness as it is believed that these two traits are predictors of leniency. The data analyzed revealed (1) a significant correlation between the raters’ score on agreeableness and the scores they assigned to the translations, and (2) a significant negative correlation between the raters’ score on conscientiousness and the scores they assigned to the translations. This proved that more agreeable less conscientious raters are prone to be more lenient. The value of Cronbach alpha also revealed that more conscientious less agreeable raters were more consistent in their assessments in both rounds of scoring. This, however, turned out not to be the case for less conscientious raters.
