I wanted to trial No More Marking as a tool for formative peer assessment.
Students were given an exam type question on radiometric vs relative dating of fossils. They were given 10 minutes to answer the question and submit their answers on Moodle, our Learning Management System, as a PDF file.
I logged in as a student who was absent on the day and submitted what I considered a model answer to the question.
When all students submitted their answers I downloaded the PDF files out of Moodle and uploaded to No More Marking. This took less than 5 minutes. Students were completing another task while I did this.
I emailed all the students a judging link: https://www.nomoremarking.com/judges/reg/yeMWKCsRWoPgCDMEP
You can use this link to judge some of the answers.
At this point, students expressed the usual concern. “I don’t want anyone to read my work.” I explained that the beauty of the No More Marking system for this type of task is that marking is completely anonymous. The task was done on the computer so students can’t even be identified by their handwriting. There are no names attached to the task so students don’t know whose work they are judging except when they occasionally have to judge their own answer against an answer from their peers.
The judging process was explained to students and they started making judgements.
I should have projected my model answer on the board at this point and would do this in the future. I didn’t do this as I was actually interested in how my answer would stand up in the judging process.
I projected the judging statistics on the data projector and it immediately highlighted a couple of issues. The first was how well most of the students were doing making their judgements. Most were carefully comparing answers and the reliability figures were quickly rising.
Unfortunately, a couple of students were obviously not taking the judging seriously. 40 decisions in 4 minutes was a giveaway. When the students realized their lack of care was visible to all, they immediately began to take the judging more seriously.
Like the staff, the students were very keen to see who was the “most consistent” judge.
I also noticed immediately that there was a very high correlation between consistency of judgement and scaled score of the student’s response. The students who could answer the question well could also judge the quality of the answers well. Students who couldn’t answer the question well struggled with the judging. That is why in future I would project a model answer to assist these students in the judging process.
I removed one student’s judging from the results. You can see the effect of that on the graph above. The number of judgements decreased but the reliability increased. She was clearly clicking and not reading but this was easily rectified.
Students were keen to see the results of the judging.
Much to my disgust, my model answer was judged second best in the class.
Note for the future: Showing the scaled score to students live is pretty soul destroying for the student at the bottom of the ranking. They get a scaled score of zero. I won’t do this in future. This students score was certainly not zero. She knew something about radiometric dating and relative dating. If I were making the responses out of 10 she would have scored 4/10.
I would like to learn more about how I could use NMM to actually generate scores to use for summative assessments. So far everything I have found out about NMM I have done with no reference to the help screen or manual. The program is very intuitive and I am one of those men who doesn’t like to ask for directions.
I am now “at my point of learning need” and want some more professional learning.
I asked the students if they felt it was a good learning activity.
- “I had to read about the types of dating about 20 times so I think I’ll remember it.”
- “I can see the level of detail I need for a really good answer.”
- “I need to explain things better.”
Good learning activity or not?
16 votes yes / 1 vote no.
High praise indeed from the end users.