Christopher X J. Jensen
Associate Professor, Pratt Institute

How Moodle allows you to remove your bias when grading quizzes

Posted 10 Oct 2016 / 0
2016-10-10Image of students taking an online exam at the turn of the century courtesy of Michael Surran via Wikimedia Commons

I think that any teacher who is honest about their relationship with their students realizes that they are not completely impartial. Given how much you can get to know your students — especially if you require students to do meaningful writing and run a classroom that’s full of robust interaction — it is nearly impossible not to fall into preconceived ruts about the performance of different students. The student who consistently participates in class and seems to grasp material gets the benefit of the doubt when their answer on an exam is a bit incomplete. But on the same question, the student who has struggled throughout the course is less likely to get that same benefit of the doubt if his answer is incomplete. I try really hard not to allow that sort of bias to creep in, but I actually don’t think that “trying not to be biased” works all that well. As a human being, I am inherently a social Bayesian: every interaction with every person I know is judged based on comparing the latest interaction with all my prior interactions. So even if you think that you are beyond other forms of bias — which I would never be so sure of myself about anyway — you probably are liable to bias based on previous experience with your students. This means that there’s an inherent tendency for your students’ assessments to snowball: poor assessments tend to lead to more poor assessments, laudatory assessments tend to lead to more laudatory assessments. That doesn’t sound very fair!

Luckily, there is a pretty good way to eliminate the bias that comes with identifying each of your students based on your prior experience with them: just eliminate their identity when grading. This can be done with pretty much any form of assessment unless you are giving many written assessments and have already become familiar with the handwriting of each of your students (this has happened to me before!). But online assessment is particularly amenable to removing student identity as a means of removing instructor bias, because typed-in answers lack most of the identifying clues needed “remember” the history you have with each student. I suppose that different students do have different writing styles, but unless you have some savant-like capabilities of identifying writing style, this likely won’t be an issue.

For a long time I have known that this sort of bias was a problem but haven’t done anything about it. So today I finally got annoyed enough with myself to go searching in Moodle (the platform for my institution’s Learning Management System) for the way to grade my online exams without knowing the identity of students. I use the quiz module for my exams, so I was looking to take advantage of Moodle’s great grading functions without being able to see who wrote which answer. At first I went looking for Blind Grading, a function that works for the assignment module (see this article for details on how to grade Moodle assignments without revealing student identities). But there’s no blind grading option on Moodle quizzes, which surprised me.

As it turns out, you can accomplish blind grading on Moodle quizzes by messing with the Permissions of your quiz. To edit the Permissions, you need to access the administration panel of whatever quiz you want to grade blindly:

moodle_quiz_settings

You then need to scroll down to the section of the Permissions that pertains to the Manual Grading Report. You’ll see a section that is called “See student names while grading”:

moodle_quiz_permissions

Basically, you want to remove your role from this list by hitting the white “X” in the red circle. Once you remove your role (and the role of any other graders in your course), when you manually grade essay questions in the quiz you can’t see the names of the students. As set up above, I can see the students’ ID numbers but not their names, but if you want to see no identifying information it appears that you can do this as well.

The only thing I don’t know about this setting is whether you need to be a Course Manager to change the Permissions. I have manager status on all my courses, so I have the ability to make this change. But when I assume the role of just a teacher, then I lose the ability to alter these Permissions. How different roles work on Moodle is going to depend on how your institution’s IT folks have set up Moodle. But if it’s a global characteristic of Moodle that only Course Managers can mess with these Permissions, here’s another great example of why every lead instructor should have Course Manager status; there are just some things that you can only do if your IT friends set you free.

I am very excited to see how different it feels to grade blindly on all my exams and perhaps on all my graded online writing assignments. The one thing that I am balancing on the use of this feature is the positive benefits of knowing the students’ identities. Clearly on higher-stakes summative quizzes it is only fair to grade blindly. But on lower-stakes formative quizzes, it is sometimes valuable to get to know students based on how they write. I will have to play around with the right combination of grading blindly and grading with my eyes open to figure out what best serves my students. But at least I now know a failsafe way to eliminate biased grading on Moodle quizzes.

A Major Post, Assessment Methods, Ethics, Learning Management Systems, Teaching

Leave a Reply