We are in the season where boards of examiners are meeting to decide the classifications of students. Data on the performance of students is carefully brought together, and a set of rules applied to render three or four years of work into a single classification. There is a parallel with the TEF – a single classification for the teaching – at least the undergraduate teaching – of a whole university. Sadly, many of the same problems apply.
One of the complaints about the degree classification system is that it presents a cliff-edge between grades. Even ministers have backed the elaborate mechanisms put in place to present the HEAR and GPA alongside the honours classification. TEF will also have its extra information, but the first focus will be on the three grades: gold, silver and bronze.
If any import is attached to these grades then the cliff-edge will really matter. It’s very helpful that ministers have explicitly ruled them out of bounds in the visa system but other uses apart from fee setting will be found in the future. As with a degree system, the difference between the very lowest of the silver and the very highest of the bronze will be tiny. Here the sorities paradox must come into play (see my discussion of that in terms of visa rules here). Universities who excitedly claim they’ve moved 10 places in a league table know that they are only fractionally different from the peers they moved past – but that may take them over a threshold.
The good news (and simultaneously the bad news) is that the TEF will be published with the metrics data, the submission and the panel’s reasons for the judgement. In my degree classification analogy; not only do we get the degree class, but a transcript (the metrics) and the outcome of the viva (the submission). When the data is released to both universities and the media, there will be a frantic scramble to discover who has a ‘good’ silver, or who only scraped their gold. More fun, we’ll surely be able to tell whose written submission pulled them up, left them where they were – or conceivably pulled them down (viva voce exams tend not to have the scope of reducing your grade – but the statement might do that).
The TEF gets described as a new ranking – there’s no way the ranking business will let the sector get away with three grades. All those data will get puled apart to create a GPA for the sector (no doubt with new means of stretching the data).
Now that the fee-level jeopardy has been pushed into the future, some of the jeopardy has been reduced, but we have already seen signs of universities reacting to TEF by changing the things that contribute to the sets of proxy data, not by being better at teaching. It’s easier to get students who will do better in the TEF, than get better outcomes for them – that’s certainly the context of some comments about Manchester’s redundancy round. If so, we’ll have the most awful consequences of accepting proxy data for assessing teaching excellence.
I should say that I write this from a Rawlsian veil of ignorance. I don’t know the metrics data of any university, what that might have indicated their award would be, or what they tried to say in their submission. I know that no rational university ought to be pleased with the situation we will find ourselves in when TEF is published. Even if they’ve just done enough to get a silver…