Using Comparative Judgement to Assess Primary Writing

Last year we made the collective decision, as a school, to move away from using descriptor-based assessment for Primary School writing samples.

This was based on our research and on our experience of using Comparative Judgement during the Year 6 Sharing Standards trial in March 2017. I previously wrote about our experience of that trial, here and here.

In this post, I detail our experience of setting up and leading a whole school Primary writing baseline task in September 2017.

The Assessment Task:

In discussion with the English curriculum leader in the school, we decided that the first task of the year should be a narrative. We thought that every child in the school would have some pre-existing knowledge of a basic story structure and would be able to respond to a story prompt. We wanted the assessment prompt to be as similar as possible across the school to allow for a fair comparison of the writing samples but we did alter the prompt slightly to meet the needs of the relevant age groups. With this in mind we divided the task up into 3 sections:

  • A Year 1 and 2 task based on a picture prompt:

Year 1 & 2 writing task

  • A Year 3 and 4 task based on a picture prompt:

Year 3 and 4 writing task

  • A Year 5 and 6 task based on a written prompt:

Write a story which ends with the words: I couldn’t believe my eyes.

As a result of setting 3 different tasks, we felt that it would make sense to divide the whole school judging session up into the 3 tasks so that teachers could focus their judgements on the year groups they taught in. This would allow teachers the opportunity to gain a specific understanding of the writing issues to address in their relevant year groups. However, we also wanted our teachers to benefit from the experience of moderating scripts from across the age ranges, Year 1 to 6 so that they could also gain a better understanding of the needs of our young writers across the school.

We were able to achieve these aims by linking the 3 different tasks together for moderation purposes. Essentially this meant that during the judging process, for every 5 judgements a teacher made, 4 would be related to the age phase they worked in and 1 would be drawn from outside their year group based on the other tasks.

Please see the relevant information from nomoremarking.com here, on how to set this up.

Running the Session:

Once all the children in the school had completed a writing assessment sample and these had been uploaded to the website, it was time to complete a judgement session. We scheduled this to take place during a regular staff meeting slot so that all the teachers could attend and all the judgements could take place at the same time, allowing for discussion and reflection afterwards.

Prior to the meeting, Daisy Christodoulou sent us a powerpoint template that provided a structure for how a session might be run. This was useful and cut down the time required for setting the meeting up. At the end of the meeting, I asked staff to engage in a reflective discussion based on the 3 questions below:

  • What were the overall strengths and weaknesses of our pupils?
  • What did you think of the 20% of scripts from other year groups?
  • What’s the highest priority action for the next unit of work these pupils study?

Whole Class feedback:

Staff were asked to complete the whole class feedback template at a time of their choosing after the meeting had taken place. This allowed me time to download the results from the website and then send on to the teachers. This gave teachers time to reread their scripts in depth and reflect on the results before completing the template:

Whole class feedback template

While CJ is a summative exercise, the template certainly focused teachers’ attention on the formative aspects of their students’ writing. Teachers could then take the time to plan effective instruction to meet these general needs in the classroom.

More effective use of the teacher’s time:

Comparative judgement not only significantly improves the reliability of teacher assessment, as explored in my previous post, but it also saves teachers a substantial amount of time, freeing them up to focus on what really matters.

Below is an estimation of how much time teachers saved using comparative judgement as a summative assessment:

Time saving picture

This was time that they could now use more productively to reflect on the results of the assessment and plan instruction to meet the needs of their students.

Another benefit of using the whole class feedback template is that we noticed how specific the feedback from our teachers became in relation to the next steps they were identifying for their students. With our previous form of descriptor-based assessment, teachers were drawing next steps from these types of generic descriptors:

  • Can produce work which is organised, imaginative and clear.
  • Can usually use correct grammatical structures in sentences.
  • Can develop and extend ideas logically in sequenced sentences.
  • Is beginning to use paragraphs.

Compare that with the type of feedback that teachers generated based on the comparative judgement session:

  • Tense – varying between first person and third person.
  • Upper and lower case letters in the wrong places, especially Bs.
  • Many of the children have a misconception in relation to our/are.
  • Only 6/19 students showed awareness of using paragraphs (with only 2 out those 6 students forming paragraphs correctly throughout their writing).

Results:

Almost immediately after the meeting was completed we were able to download the children’s ranking results based on a scaled score. We organised the rankings by year group and by class to help teachers reflect on their students’ current level of ability. This information could then be used to inform the whole class feedback template mentioned above.

A few days after the judgement session, Dr Wheadon got in touch and provided us with plot graphs that gave us a great visual representation of the results from a whole school perspective. An example of one is provided below:

Results.png

The results showed that, in relation to writing, there was generally a good level of progression from year group to year group across the school and that our students seem to make a substantial amount of progress in Year 1. As well as helping us to establish general trends across the school, the results also highlighted to us those individuals who had produced pieces of writing that were judged significantly above or below the standard of their current year group. Of particular note was a group of under-performing boys in Year 5. The results provided us with some interesting information to reflect and follow up on.

Reflection:

One minor issue that we encountered during the process related to the setting up of the task. The website itself, nomoremarking.com, contains a great deal of information that details how to set up a comparative judgement task and this information is constantly updated for users to access. However, trying to locate the exact article I needed was not always easy but this was not generally a problem as Dr Wheadon and Daisy Christodoulou were extremely helpful and did not mind replying to emails or arranging time to talk with us over skype.

The whole school comparative judgement session prompted a lot of reflection on our standards of writing. Just the process of being able to look at samples from across the school and see the journey the children go on, as they develop from making their first marks on a page to structuring creative and descriptive sentences, is hugely beneficial.

Not only this, but the process also saved our teachers a considerable amount of time and enabled them to make more reliable judgements about the baseline writing samples they were assessing. Allowing staff time to discuss and reflect on the judgements they had made at the end of the meeting and adding the whole class feedback template into the process also seems to have allowed teachers to create more specific and relevant targets for their students.

 

 

2 thoughts on “Using Comparative Judgement to Assess Primary Writing

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s