Blurred 2020 Vision: SQA exam results
In my day, the exam results came out during the Glasgow Fair and we were always on holiday. I remember lining up at a phone box in Torquay to call my older sister who was at home. She opened the brown envelope and read out my results to me. Luckily, I got what I deserved. I got what I had worked for.
Now the exam results are issued later and can be delivered electronically wherever you happen to be in August, but the anticipation and worry for candidates is the same.
Even as a teacher I felt the build up of tension and anticipation waiting on my class, then my departmental and then, as HT, my school’s results. We all wanted all young people to get what they deserved, what they had worked for.
2020 has given us unprecedented times, not the least of which the cancellation of the entire SQA examination diet for schools and colleges.
This announcement in March 2020 was the first time the exams have been cancelled since the system was put in place in 1888. At the time of the announcement, which some felt was premature, the Education Minister, said a model would be put in place to ensure that “young people in schools and colleges who were unable to sit exams would not be disadvantaged in any way”. The Scottish Qualifications Authority (SQA) said work had started "at pace" to develop an alternative certification model.
This alternative certification model amounted to asking teachers to submit estimated grades for each candidate in each course, SQA acknowledged that teachers and lecturers were best placed to have a strong understanding of how their learners perform and, based on their experience and the evidence available, what a learner would be expected to achieve in each course. However, SQA would have the power to change teacher decisions based on schools and colleges’ history of estimating grades and pupils’ previous grades.
SQA said that “an estimated grade is not just the result of one prelim or one project, but is an overall judgement based on all activity across the year.” Therefore, during the height of COVID 19, teachers worked hard to meet the May 29 deadline set by the SQA and submitted their holistic grade for each young person. This grade was based on prelim results, class assessments, teacher professional judgement and moderation at department and school level. To paraphrase a few headteachers I know, “nothing left the school without us having oversight of it”.
In the intervening period, while concerns were raised and questions asked and warnings given about potential consequences about the proposed use of historical estimating of grades by schools, there was the sound of silence from the national examination body.
On results day, 4th August 2020 there was an increase in the pass rates across the board. This is to be celebrated. Yet there was widespread anger and frustration across the country and stories of young people devastated by their results.
So with “ anger and dismay coursing through the country,…… the sense of injustice felt by so many teachers and young people” (H Hepburn TES 05/08/2020) we can only conclude that the alternative certification model has failed to deliver on the promise that “young people in schools and colleges who were unable to sit exams would not be disadvantaged in any way” (J Swinney March 2020).
So, what, if anything went wrong? Firstly, SQA claimed when questioned by the Scottish Parliament Education and Skills Committee in May 2020 they would "undertake a moderation exercise nationally, using a range of data, discussion and review". Yet they carried out no transparent discussion or review with schools or teachers. This in a Scottish Education system which purports to promote empowerment.
Secondly, the methodology used by SQA, which was published on results day, shows that the historical attainment data of individual schools was the only basis for changing teacher estimates. SQA used evidence about previous candidates’ performance, estimates by previous teachers and a school’s historic attainment data as comparators while awarding the 2020 cohort their grades. A teacher writing in TES Scotland today asked: Is it fair to adjust based upon other classmates' results rather than nationally? Is it fair to adjust based upon the school subject result history? Is it fair to adjust on overall school attainment or SIMD (Scottish Index of Multiple Deprivation) profiles? Is it fair that smaller cohorts are adjusted in a similar way to larger cohorts?[1] The answer to each rhetorical question is a resounding NO.
Thirdly, the outcome of using the methodology has, not unexpectedly, resulted in apparent disproportionate downgrading of grades for candidates living in the poorest areas. The SQA table below shows SQA lowered the estimated grades for those living in SIMD 1 and 2 by 15.2% while the estimates for those in the most affluent areas were adjusted by 6.9%.
SQA had been forewarned of this statistical outcome. Barry Black, University of Glasgow said “Any system would have needed elements of moderation, but the full reliance on historical attainment data has caused these issues – as [the SQA] were warned from the outset.”[2]
Equity is a cornerstone of current Scottish Government education policy but this ‘alternative certification model’ has highlighted the inequity in the current system which has been further embedded in this changed approach. As another anonymous teacher said: “It is an education system which still favours the affluent; an education system in which young people who grow up in poverty are still disproportionately more likely to leave school with no qualifications, and with lower attainment in the qualifications they do get”3
Lastly, the estimates submitted by teachers and schools, who it has been stated and restated several times, show significant increases on previous exam results data. What we don’t have to hand is teacher estimated data for the previous years. So, in the Table A13 below, while the estimated proportion 86.7% of A-C passes for males sitting in between the 72.2% in 2019 and 76% for 2020 results looks out of place, we do not know what was estimated last year.
It would be foolhardy to lay the blame for 2020 SQA results issues at the door of teachers by claiming they have overestimated on some grand scale. Teachers from time immemorial have, thankfully, always erred on the side of the young person in front of them, the teenager who they have invested time in and nurtured. However, in my experience this is usually at the margins, at the grade boundaries. As a consequence, teachers are not surprised although disappointed for the candidate, when some awards get adjusted at these margins. However, the results being reported yesterday and today cannot be explained away in this way. There are numerous examples of candidates being downgraded across 2 grades and 4+ bands.
Leaders in Scottish Government have been quick to point out that sticking with all teachers' estimates would have led to pass rates far above those in previous years and therefore would have undermined the credibility of the 2020 exam results. Here’s the news, folks: the credibility of the 2020 exam results is compromised anyway. This was an exceptional year and we could have had exceptional responses to it. We could have been bold. We could have been courageous. We could have re calibrated the inbuilt inequity in the system in 2020 for years to come.
Setting aside the human fall out from the cancelled SQA exams and all that followed we now move on to the next huge challenges for the system in Scotland.
Pupils and staff return to school full time on 17th August while COVID 19 is still in our communities. They will need real support and assistance in making such a move safe for all.
Teachers will be under more pressure than ever before about entering appeals for disappointed SQA candidates (and there will be 1000s of them) before the UCAS deadline in August too. It should not be forgotten that the young people who will need the greatest support and nurture on their return are the same families let down by the 2020 SQA moderation proves. The pressure in these schools in the poorest communities will be immense.
SQA MUST ensure the appeals process is transparent and uses “range of data, discussion and review” so that individual young people get the result they deserve and not a derivative of previous data
We need proper blended learning models and resources to ensure continuity of teaching and learning for all when the next school closures happen – even locally
Finally, we need an actual alternative assessment and certification model for the next iteration of a pandemic. There is sure to be one.
[1] https://www.tes.com/news/sqa-results-day-2020-am-i-dishonest-am-i-unprofessional
[2] https://www.scotsman.com/education/scottish-pupils-risk-being-punished-schools-past-failures-2544376 Barry Black
[3] https://www.conter.co.uk/blog/2020/8/4/not-credible-sqa-class-and-education
