The outline of the grade awarding process for 2020 released 20th March by the DfE has revealed some information about the external validation process. The significant role of teacher predictions will bring tough emotional and logistical challenges as well as the potential for friction between teachers, subject leaders and senior leaders.
A very important part of the submission of teacher predicted grades will be internal verification and standardisation. No-one will want to submit grades that are at risk of being pulled down because of inconsistency, or because of unrealistic or unjustifiable grade predictions.
Once centres have internally verified and submitted the subject grade predictions for their students to the exam board there will be a process to calculate the final grades awarded to students. Using the limited detail about this process so far and despite the unprecedented situation, we do have examples of current practice by JCQ, Ofqual and the exam boards and enough information that points us in a likely direction. This can be used to help us to prepare rigorous internal verification processes.
Early preparations around your current data and planning for your internal verification processes will support with clarity, consistency and confidence of the teacher grade predictions and ease those difficult conversations with staff when internally verifying. We hope this also improves the accuracy of grade predictions and avoids the risk of having whole cohorts for a subject and/or centre adjusted down by the exam boards.
Having access to the data whilst teachers and leaders are working from home will make this process even more challenging, especially if different spreadsheets and inaccessible MIS systems are relied upon for deciding and verifying teacher grade predictions. This is why we are are working hard to provide access to our online Pupil Progress platform to everyone, with exam board specific trackers ready to use.
What we know from the DfE
We know that teachers will submit their judgement about the grade that they believe the student would have received if exams had gone ahead. In reading the Press Release by the DfE on March 20th, there are some key points worth noting about the way these submitted grades may be externally adjusted by the Exam boards using other data sources, including;
- prior attainment
- the distribution of grades so that it follows a similar pattern to that in other years
It is also worth noting that BTEC Internally assessed units, Pearson (Edexcel) stated on 20th March that they will “continue with standards verifications this year and will confirm teacher assessment for units already completed” (Rod Bristow President, 20th March). The format and approach to this is likely to change as more staff will have to stay at home.
Distribution of grades
Every year the grade boundaries are adjusted to ensure that a fixed percentage of students in each year group achieve a similar distribution of grades. JCQ and Ofqual work closely with the exam boards to ensure the statistics each year are relatively consistent. JCQ publishes the percentage of students achieving each grade each year and each exam board also publishes theirs for each individual subject-specific qualification, for example see AQA's results statistics. Ofqual publish these annual grade distribution statistics per subject across all exam boards, for example:
This year is no different as stated by the DFE; “We will also aim to ensure that the distribution of grades follows a similar pattern to that in other years” (Press Release by the DfE on March 20th). Therefore submitted grades will be adjusted by each exam board overseen by JCQ to fit the usual grade pattern seen in each individual subject qualification across the national cohort.
Just to be clear, we are not saying that each centre should follow the national grade distribution, but that it will be used across a national cohort and will show exam boards how much they need to adjust grades awarded for a particular qualification. With the internal verification process it is more important to look at the role of prior attainment.
The role of prior attainment
There are essentially three prior attainment data sources that can reliably be used in this process:
- KS2 data combined with GCSE results from previous cohorts
- School-based results from previous years
- For WJEC A*-G GCSE’s and vocational courses such as BTECs, the students are likely to have already sat externally assessed units or modules
In each case, the prior attainment is likely to be used in conjunction with a yet to be defined tolerance limit. We see examples of this practice in the moderation process of internally assessed units, non-examined units and coursework. We are unlikely to find out these tolerance levels until after the grades are awarded (if at all!) to rightly avoid game playing. The prior attainment data could be used at subject-level and at school-level to decide on whether the submitted grades should be accepted or further investigated. If they are outside these tolerance limits then the whole cohort’s predicted grades may then be adjusted up or down by the exam boards.
Use of KS2 Data
As we know, Progress 8 is calculated based on KS2 data. As the DfE will have KS2 data for the majority of students from a centre (Your School), they can use this with the attainment data from previous cohorts to estimate the likely outcomes for students with a particular KS2 prior attainment level.
We all know that at an individual level this is very unreliable, and this is evidenced by the fact that the DfE only publishes Progress 8 if it is measured over a larger cohort size. Therefore this is most likely to be used at a school level to see if the grades across all subjects come within the tolerance limits.
Therefore, prepare by making sure your data sheets have KS2 data available as a marker to check against. For example, if a student is in a low KS2 prior attainment and yet is predicted a grade 7+, although possible, statistically this would be an outlier and therefore should be internally quality assured and justified.
We would strongly advise not to rely on target grades that are based on KS2 for internal verification. In the majority of schools, these are based on the highest progress levels statistically seen in students and are purposely set to a challenging level to push students and teachers to achieve highly. Therefore there is a risk that it would result in centres grades being inflated and therefore result in the exam boards adjusting them down.
Use of exam centres results from previous years
Each exam board will have attainment data for each exam centre for each subject and qualification (GCSE, A-Level, BTEC Tech Award, etc) when they sat it in previous years. It is possible to compare the school’s cohort grade predictions for 2020 to previous years cohorts. This may serve as a marker for further investigation for a particular subject within a centre if they are dramatically different. This combined with KS2 prior attainment may also serve as an indicator for whether exam boards will need to moderate marks up or down.
This was confirmed by the DFE, stating; "Ofqual will consider carefully how to ensure the process is as fair as possible, which is likely to include considering measures that reflect how much progress a student would have been likely to have made at the school they are attending." (DfE Guidance, Q6, March 20)
Therefore, we would advise that you prepare by making sure you can see the percentage cumulative grades predictions per subject achieving 1+, 4+ and 7+. If it is significantly different, then this may warrant further investigation before submitting.
Our tracking software enables these cumulative percentages to be seen live at a subject-specific level. In light of the extreme pressures teachers are under at the moment, we are offering completely free access and use of our tracking system to support you with this.
Completed units & exam papers
Vocational courses should in practice have finished most internally-assessed units and have internal grades for these units. BTEC courses have stated they will still continue with a reduced version of the verification process (Rod Bristow President, 20th March). It would make sense that other vocational courses will be in a similar position too.
Make sure all the students’ work is available in case of being requested for verification. Make sure all the data for completed units and incomplete units is as up to date as possible. Pupil Progress can give you free support with collating the data using our subject-specific trackers and grades to support an accurate grade prediction process.
A large number of courses have exam series in January and March, such as Cambridge Nationals, Cambridge Technicals, BTEC Tech awards, BTEC Nationals and some WJEC A*-G graded courses. In many cases, students will have been re-entered for the exam units in Summer 2020 if teachers felt they had a likelihood of achieving a higher grade. If they have not been re-entered, the grades for these units are likely to be used. The grades currently achieved are going to be the best indicator for the overall grade. This is also common practice seen in the guidance used by JCQ when dealing with Special considerations (A guide to the special consideration process 2019-2020, JCQ). DfE have also indicated Ofqual will draw on this experience "of using moderated teacher assessment to award grades to a number of students, such as those who fall ill immediately before an exam." (DfE Guidance, Q7, March 20)
Make sure your tracking indicates which grades are from externally sat exams and use it to compare with the overall subject grade predictions.
Further support and preparation
In our previous blog on Preparing for Teacher Predictions we outlined the way subject leaders can prepare their raw data ready for grade predictions. To add to that we would also recommend that leaders makes sure that teachers and subject leaders have access to:
- Prior KS2 Attainment data
- Previous year cumulative percentages of grades e.g. % 1+, 4+, 7+ for GCSE, E+, C+ A+ for A-Level, L2 Pass+, L2 Merit+ L2 Distinction+ for BTEC courses
- Data from previously sat exam units and internally assessed units
If you need support with collating this information and calculating the % cumulative grades instantly, we are offering completely free access and use of our exam-board and subject-specific tracking system to support you with this.
We know that this is a very challenging time for all teachers and leaders. There will be a mix of emotions and many sources of friction and causes of anxiety. All blogs relating to the Teacher Grade predictions in 2020 are linked to our article COVID-19: Supporting Teachers through the Crisis.
We will also be releasing a blog once further guidance comes out. If you would like further information, you can sign up to notifications related to this blog series.