“Well, she deserves a 7”- Reducing unconscious bias

Author: Brett Griffin

Category 2020 Exam Grades

We all know and understand the importance of the task that teachers are now faced with, this is not something that needs to be dwelled on. The announcement to cancel exams, but still award grades was made and the decision has now been finalised. Not only will centres be responsible for submitting grades but also now how this is going to happen. So, in short, this is the time for action and time to be as well informed as possible before starting the process.

The purpose of this blog, and the connected series of blogs, is to open thought processes and mindset so that teachers are as well prepared for the task as possible. The specific focus of this blog is the ways in which we believe that you can reduce the impact of unconscious bias when making your judgments by making a well-informed decision based on evidence and historical data. This will look at not only tackling bias when predicting grades, but also bias when ranking the students before submission.

To provide context to this blog, this is specifically addressing ‘Unconscious Bias’. We know that teachers will be as professional as they possibly can be with their judgements, it is to highlight and consider the bias that we are unaware of and provide practical solutions to reduce its effect. There are a number of studies that evidence the existence and presence of unconscious bias within the educational setting (an article by Pran Patel, 2020 Exams Result and Bias is a great starting point) and it’s impact on predicting grades. 

Our article outlines a clear methodology that not only helps you complete the task in hand, but it is a process that will help reduce unconscious bias as a natural byproduct.

Click here if you want to download our full guide to read later   

Avoid making judgements based on behaviours

When this surfaced as a problem I immediately defaulted to the mindset of when I was teaching, what would I have done as a teacher? What would I do as a senior leader that needs to support and quality assure this process? 

My immediate response and the solution that instantly screamed at me was an approach that would focus as much as possible on the evidence of skill and or knowledge, one that would avoid making decisions based on any behavioural characteristics whatsoever. Ofqual have made it really clear that “It is important that the judgements are objective, and they should only take account of evidence about student performance.” (Ofqual, 3rd April) [our emphasis].

Examples of behavioural judgments to avoid; 

  • Attitude to learning 
  • Behaviour in class
  • Concentration levels
  • Your relationship with the child
  • How well they pay attention in class
  • How well they work with others
  • How many times that student completed your homework

By collating the data on how students have performed in each component historically, with regard to the specific skill and or knowledge that is required for success in a specific section or criteria you are making judgments based on evidence, not behaviours. Being this granular will help you to reduce the focus on how you feel their behaviour would have influenced their outcomes. 

Click here to find out more about our free online tracking system to support  this whole process

Example 1: how well did ‘Claire’ do when previously presented with the opportunity to answer question 5, worth 24 marks on the English Language Paper? If the last 2 times that she achieved 16 and 18 it would be reasonable to suggest that she could have achieved 17-20 in the final exam. If you are able to break down the marks that make up the 24 to justify the mark of 17-20 then even better! This is a decision based on her ability to answer the question, not her behaviours or your relationship with the student. 

Example 2: How did ‘Adam’ perform when he previously had the opportunity to answer section B in the AQA GCSE Drama Exam paper, comprising 44 marks?  This would be an example that, where possible, uses an even deeper breakdown of the section, for example; ‘are there common patterns with the questions answered that justifies an evidenced based judgement on that particular question?’. 

The two methodologies above, when applied across the whole course will provide a judgement on an ‘Overall total mark’. It is the total mark that is then applied to the grade boundaries, the actual number (Overall Total Mark) will tell you the grade. Judgement based on behaviour will not tell you the mark, it will be a grossly inaccurate grade based on a holistic “feeling”. 

Avoiding bias when ranking

We have discussed the challenges of ranking students in more detail in an article Facing the Challenges of Ranking your Students. Here we are specifically on avoiding bias when ranking the students after you have predicted the students final grade. For those of you that aren’t aware  once the grade has been decided all students must be ranked in order (No one student can be ranked at the same level or the submission will be rejected). So, for example, all students that are all predicted a grade 5 for a single course and subject, must be ranked in order of confidence that they would have achieved that grade.

The good news, if you have been granular in your approach to predictions, you will find yourself in a beautiful position. One in which a very large percentage of the students, if not all of the students you teach, will have a different final grade and total course mark that has informed that grade. This means you simply put them in rank order of the ‘Overall total mark’. If two students have exactly the same grade and total course mark either from the same class then you simply revisit the individual course sections for each of the students with the same ‘Overall total mark’ with a higher level of scrutiny and adjust marks where you may have been either too harsh or lenient with a particular mark in a specific section. This can and should be done in discussion with the teacher of that student involved. The breakdown in unit grades will help the teacher that hasn’t taught that student to make a well-informed decision on the judgements made.

The question I must ask is: if you do not use a ‘Overall total mark’ of the course to calculate the grade and rank the students then how would you decide whether a student with the same grade deserves to be above another? The answer is simple, you will be forced to use behaviours as your evidence and will rely on the advocacy skills of individual teachers. This will cause the application of unconscious bias, there is no other way of making that decision fairly, it’s just simply not possible.

Click here to find out more about our free online tracking system to support  this whole process

The truth (data) will out, 

I will actually make a bold suggestion, that if you focus on the marks within each unit and the evidence of how students have performed previously in each individual section, with as much detail as you possibly can, you will find that once students have been ranked there will be a student or students that have performed better in your judgements than others that you feel do not deserve to be above certain other students. 

However, this is the exact point, the exams that students sit don’t assess how well a student has behaved for the last two years, how much a student has paid attention in class or how they have behaved for you. An exam will often not equate to the grade a student ‘deserves’, this is too subjective. An exam specifically tests a students skill and/or knowledge application across a variety of assessment objectives. The harsh truth is that there are plenty of students that don’t work as hard, or don’t behave as well, or are unconsciously perceived to be of a lower ability because of their gender or ethnicity that retain and learn more than we thought they are, would or could and therefore exceed our expectations in the final exam. It is this that must force us to behave as much like the exam process behaves as possible.

Naturally, as a generalised rule, those students that haven’t paid as much attention, that have not worked as hard as others and those that have not concentrated as much as others will be the ones with a lower total course mark. The very important point is that you must let the data tell that story, not judgements based on student behaviours. 

It must be recognised that it is almost impossible for humans to behave without unconscious bias, (after all we aren’t aware of what specific bias we may be applying!) but following a system, that has a clear methodology and structured process will dramatically reduce this influence. This will serve for a far more honest and fairer prediction and grade ranking for the students.

 

If you want to find out more about our free tracking to support you through the 2020 Grade submission process, click the button below.

Click here to find out more about our free online tracking system to support  this whole process