‘Grade Inflation’ – a straw man argument

The Higher Education sector needs to explain what’s going on with this trend we’re calling ‘grade inflation’. HESA data show that the pattern of achievement in honours degrees is now very different to how it was a generation ago. This plays into a narrative about university education that’s really unhelpful.

We need arguments and explanations of what’s going on. Here’s Liz Morrish offering a cogent perspective on the change last summer in this post. Graham Virgo offered some of these arguments to the Telegraph (note that his explanations become downgraded to ‘claims’).

Inevitably the Telegraph turns to Professor Alan Smithers, the Eeyore of academic standards, for a view. He’s not impressed. He says:

“It sounds to me like a narrative designed to bat away criticism of what is an obvious problem,”

“It is possible to come up with an accepted grade distribution. Within the sector you could say that in any university a set proportion would get a first or 2:1.”

This is absolutely not the answer to ‘grade inflation’.

Higher education uses criterion referenced assessment. We grade students’ assessment according to criteria – setting out our expectations and seeing if they meet them. Our criteria have changed as assessment has changed and as courses have changed – we mustn’t pretend they haven’t- but that doesn’t mean they are ‘easier’. Crucially, as Morrish points out, we are all better, staff and students, at understanding those criteria and more students are getting marks in those higher grades. This is where the explanation must focus.

What Alan Smithers is proposing is norm referenced assessment. This fixes a grade distribution and then plots students against it. Our national examination system is predicated on it; in addition to knowing whether a student had met a set of learning outcomes, we learn whether they are in a band of achievement for all kids.

I’d be happier if we stuck to criterion- referenced assessment for national exams, but the logic does work at this level. It is useful to know which students are performing best. So Ofqual has a formula, because it wants the new grade 9 GCSE to be scarce, it is capped:

Percentage of those achieving at least a grade 7 who will be awarded a grade 9 = 7% + 0.5 × (percentage of students awarded grade 7 and above)

So, if every entrant in Maths is taking broadly the same exam, you can, as Smithers says, set the proportion getting a 9. Obviously, as Smithers knows, every university student is not taking the same exam. Their assessment is run by autonomous universities running different courses with different assessment. That’s the strength of our system. A national curriculum in HE would be a disaster, so you cannot possibly fix a national assessment. So autonomous universities must continue to do their own assessment.

So, how could you possibly norm-reference thousands of different courses in over a hundred universities? This would turn into a quota system, but because we are a rather hierarchical system, this too would be monstrous. What if firsts were only to be awarded to the ‘best’ 10% of students in a university? Is that on each course, or across the university? Tough on you if your cohort of 20 in your year had two outstanding students – they’re getting those firsts however good you are. But if it’s across the university, then tough on the historians because the physicists are going to get ‘better’ marks and take more of the 10%.

Roll that argument out to universities. Should Oxford and Buckingham each only award 10% firsts? Or will the norm-referencing agency allocate differential numbers of firsts to different universities on a pre-calculated basis?

This is a series of straw man arguments, of course, but they are offered because people like Alan Smithers offer a glib response to a complex question that they must know is utterly unworkable and would be monstrously unfair. Let’s talk about how students meet our criteria, let’s accept that the classified honours degree is problematic, but let’s not pretend you could do a grade distribution system.

Advertisements

Rules for the House of Wisdom

Higher education endures. For all the talk of avalanches and market exit, there is a tremendous continuity in HE and its institutions. Chief witness to that is our ancient universities.

Part of that continuity is the ‘student experience’ – sometimes harder to capture than other aspects, but still there. My favourite tool for demonstrating this are the regulations for the Collegium Sapientiae.

It’s worth remembering that other European universities had collegiate structures; the Sorbonne became synonymous with the University of Paris, but it was a college. The Collegium Sapientiae was founded by the 1497 will of Johannes Kerer, who’d been involved in the university of Freiburg since its foundation in 1457.

The content of the regulations is broadly consistent with other sets, but these differ in two main regards. First is their comprehensiveness; they contain a fulsome description of the rules and the rationale behind them. Secondly, and this is my favourite part, they are illustrated. Most of the individual regulations come with a picture. This means they must be among the most beautiful university regulations.

The college closed, but the university remains proud of the regulations. I first spotted them on a visit to the university museum. However, serendipitously I later saw the facsimile edition on display in Mainz. Published in 1957 this is a full reproduction accompanied by a transcribed version of the Latin text and an English translation. Here are a few…

Admissions

The rules on admission are clear:

Only those who were born in wedlock can be admitted to our house. Our doors may not be opened to those who are married or who live in a state of bigamy; to those who are apostates, blasphemers or to persons of quarrelsome nature, to those who suffer from serious or chronic diseases, those who are feeble in body, to vagabonds, unchaste persons or those who have a sullied reputation…

However, there is a clear mission here:

But our House is open to those who are poor, talented and who thirst for wisdom.

Accommodation rules

There’s a full suite of regulations for the rooms, their allocation, furniture & inventories.

The President shall show a newcomer to his room. He shall also require the candidate thus selected to make up a list of the furnishings within that room, so that when he takes his departure he may be made accountable for them.

Student Conduct

The restrictions on students were considerable. Examples include:

We strictly forbid any form of revelry within our House. Neither shall our scholars indulge in drinking and feasting in inns, where every vice is permissible. We want the scholars of our House to abstain from all such vices and to indulge their youthful high spirits without the stimulus of alcohol…

As very often wickedness steals into the pure hearts of young men under the cover of respectability and hilarity, we allow of no musical instruments within our House. …

Dice, cards, and sticks for casting lots and all games of chance are forbidden. … Chess, however, is allowed.

Learning and Teaching

The college had rules to support students at university. The students had extra disputations in the college, and rules provided extra support for students who needed extra time to complete.

Continuous studying produces choice blooms, but interruptions necessarily result in a downfall. We hereby ordain that every occupant of our House shall attend his lectures, above all the normal ones, attentively and without interruption, with the help of his own book or one that has been lent him – for he who learns without a book, ladles water with a sieve. …

Penalty system

With all these rules, there needed to be a system of penalties for transgression. There’s a fine tariff system; some with specific targeted penalties but many based around the removal of wine for a specified number of days.

As an example, being caught speaking in the vernacular the punishment is loss of your eighth of a litre of wine. Entering the room of other scholars is punished by loss of wine for a week. The first failure to make your bed results in the removal of wine, but if there’s repeated offence then ‘the scholar in question shall be deprived of his bed’.

Reference

Kerer, J, 1497, Statuta Collegii Sapientiae Facsimile Edition Ed Beckmann 1957 Jan Thorbecke Verla Lindau & Konstanz

Adventures of a young academic registrar 

Waiting for the ‘major review of university funding and student finances’ brings to mind the last proper review, the Dearing Review, 20 years ago. Having also spent time perusing HEFCE circulars from the 1990s looking for details of MASN, I’ve been quite nostalgic this term.  20 years ago I was in my first year as an Academic Registrar and these were exciting times.  So I’ve looked up my notebooks to try and capture what was going on in autumn term 1997.

My notebooks record a busy term. There was the normal business of King Alfred’s College (now the University of Winchester) – professorial appointments, student disciplinary hearings, committees etc.  Layered onto that was the prospect of change brought on by the Dearing Report and the impact on the college as it looked forward to gaining university status.

Dearing

The scope of the Dearing review was huge.  In addition to the issue of fees, the report had implications for all parts of the sector.  In October Ron Dearing gave the Fawley Lecture at Southampton, framing the notion of the learning society and how higher education played a key role.  In all the talk of change, Dearing stressed the enduring purposes of higher education, closing his lecture with the same Masefield quote that had been in the introduction to the report.

‘It is a place where those who hate ignorance may strive to know, where those who perceive truth may strive to make others see; where seekers and learners alike, banded together in the search for knowledge, will honour thought in all its finer ways, will welcome thinkers in distress or in exile, will uphold ever the dignity of thought and learning and will exact standards in these things.’

By December details of the structure of the QAA were emerging, with the Gloucester location settled upon and a staffing structure agreed.  Dearing set a huge agenda for the new agency though and much was still under active discussion, including the plans to create a register of external examiners.

Quality Assessment

In addition to a re-accreditation exercise by the University of Southampton, the college was dominated by the then HEFCE teaching quality assessment process.  Using the graded profile method this precursor of TEF actually visited institutions; watching teaching and sampling work.  My notes record the arrangements for the visits.  One planning meeting genuinely started with a discussion of the state of the Gents toilet and that the whole building needed washing.

Then there was the grand theatre of the feedback meeting.  The ‘scores’ for the graded profile were announced, the goal (at that stage) to ensured the provision was approved (by gaining 3 in each section) and then to maximise the reputational gain by getting as many 4 grades as possible.    Much hung on those six scores.  Notes of one feedback meeting record the mixture of criticism – especially on the lack of self-criticism (the self-assessment was ‘clear but bland’) – with praise for high quality teaching.

Looking Forward

Ron Barnett’s inaugural lecture in November started with the cheerful claim that the western university was dead.  Helpfully, out of a ‘constellation of fragility’ it could be re-born.   If the university adopted an ethos of collective self-irony, it stood a chance.  He raised the issue of supercomplexity and how the university must create uncertainty, especially in its teaching.

Issues in 1997 are very similar: in the institution it was about matching curriculum to student demand, structuring the academic year, matching staff workloads to RAE requirements etc. Outside, the issues of funding, autonomy and relevance were being shaped.  However, Ron Dearing was right, higher education endures.

Willetts on more will mean worse

David Willetts loves Universities.  He tells us that in the opening sentence of his book ‘A University Education’.  This will be an important book, I’m sure.  It’s also unusual.  Willetts is a plausible author, a minister who, despite the extraordinary rancour of the debates about higher education in his four years, left the post generally respected, if not wholly agreed with.  Imagine how Jeremy Hunt’s big book of the NHS would go down, or a Liz Truss study of the Judiciary for comparison.

There are reviews already – Nigel Thrift in the THE and Andy Westwood for HEPI for example.  This is not going to be a review.  I do want to draw people’s attention to Willett’s robust critique of the ‘more will mean worse’ argument.  A large part of the book is spent defending why it’s worth going to university, and he takes on Kingsley Amis (and his contemporary followers) firmly.  Amis had thought that academic standards were going to be harmed beyond repair as we moved to 5% of the population going to university, now we are close to 50%.  Willetts writes:

We can concede something very important to Kingsley Amis, however. Setting aside his anxieties about their effects, he was the first writer to observe one of the great educational and social changes in post-war Britain – the shift from elite to mass higher education. It is a crucial change in the character of the university which is not widely appreciated or understood.  When we see photographs of the M1 in the years immediately after it was opened we are amazed at how quiet and empty it looks and perhaps dream of being the solitary driver cruising past Luton in 1960. The M1 was expected to carry 20,000 cars per day and is now taking 140,000 – growth on a similar scale to higher education.  But we know there is no going back and that if we were such a solitary driver we would be in a country that was actually very different and deeply impoverished compared with what we all now enjoy (or we would think an epidemic had broken out and nobody had bothered to tell us).  We don’t just have more university places and more cars – we also have more TV channels, more foreign travel, more supermarkets, more information, more books, and more films – the shift from elite to mass is one of the features of modern capitalism and we cannot and should not stop it in higher education any more than anywhere else.
Willetts, D, 2017, A University Education, Oxford, Oxford University Press pp 143-144

No doubt we will debate many aspects of the book, but let’s remember that David Willetts has set out to defend the move from an elite to an universal higher education system.  That alone makes this welcome.

 

Student petitions – the paradox of publicity

Petitions are an ancient way of gathering support for an issue. I’ve never been that thrilled by them; it’s a particularly passive form of engagement, but they have their place.

I remember that there was always a petition to sign when I was a student – I inhabited the students union and collecting petitions was a standard activity of societies of all types (I always wondered whether it was more likely that a foreign dictatorship would release a political prisoner than the university would get a new vending machine as a result of a petition).

But the routine of students standing outside the Union at lunchtime trying to slow down progress towards the sandwich shop has been replaced by the online petition. This has two main effects; firstly the things have proliferated – it’s so easy to do, you don’t even have to find a clipboard. Secondly, it’s so easy to distribute – and here a problem begins.

Students are at the leading edge of many changes in our society. Much of the activism that I remember from university is now mainstream, but then the anti-apartheid activists or hunt saboteurs were viewed as dangerous lunatics.

Now the media can seize on ‘dangerous lunatics’ to feed its constant hunger for content. One student had gathered 50 signatures this week on a petition to take Gladstone’s name off a building before the press caught hold of it. This generates news items and columnists, fitting into a narrative of student activism that has generated so much heat (think Cecil Rhodes).

Examples abound: a vegan student at Hertfordshire worries about the welfare of reindeer at a Christmas market. This is in the local paper today, but give it a ‘students against Christmas’ theme and it could go global. She might be safe, but if she was on a JCR committee at an Oxford College it would, apparently, be obligatory to cover this.

But this leaves a problem behind. The stories left online can create a false narrative. A student petition complaining about Germaine Greer giving a lecture transmogrifies into Cardiff University banning her – such that its VC found himself at Parliament’s Human Rights Committee with the free speech activists of Spiked! See the horror here: http://www.parliamentlive.tv/Event/Index/214a7bfe-da17-40f3-8022-1915a59e683a

I think there’s more of an edge to the use of these stories today. The University feels embattled within a post-truth, anti-expert narrative, which in the US is explicitly being linked to the Alt-right movement.

Students are caught between a prurient fascination with young people’s lifestyles and a horror of their supposed ultra-censorious behaviour. Flick through a list of student societies and you’ll see that they want to end sexual harassment, racial abuse, inequality, cruelty to animals, nuclear weapons, climate change, deforestation, poverty, etc. These students represent hope, yet they are mocked. They are both ‘snowflakes’ and dangerous lunatics simultaneously.

Here’s the paradox; I want students to keep on doing what they’re doing – pushing the edges of human behaviour. Their campaigns will be many and various, wonderful and wacky. I want the media to cover that – you can’t campaign in secret – but maybe stop generalising from a small campaign to to the totality to students and, especially, the universities at which they study.

Why do the University title rules not apply to those nice former footballers?

We know the Government is keen on Alternative Providers, but why have they bestowed such a large favour on University Academy 92 by allowing it to include the word ‘University’ in its company title?

John Morgan followed up on this in the THE:

Questions have been raised about the Department for Education’s decision to allow UA92 to use the word “university” in its title (the DfE has said it did so on the grounds of Lancaster’s involvement).

UA92 does not yet formally exist (recruitment is scheduled to begin in January 2018, with courses starting in the autumn of that year) and does not have its own degree-awarding powers. So, is “university” the right word?

“It is a university,” replied Mr Neville. “We’ve got permission from the DfE to be able to use the word ‘university’…The reason we have the word ‘academy’ in there is because we’ve got a 16-18 offering as well.”

How does this work?  We know that there are currently two routes to full University title.  A higher education corporation may ask the Privy Council to use the powers it has as part of the Further & Higher Education Act (FHEA) 1992; the criteria are issued by the DfE and they gather advice through a process run by the QAA.  A company also goes through the QAA process, but the DfE’s advice culminates in a ‘non-objection’ to the use of the sensitive word ‘university’ under the terms of the 2006 Companies Act.

So, Mr Neville is wrong.  UA92 is not a university.  It does not have degree awarding powers, so it cannot be awarded university or university college title.  But DfE has given permission for the use of the word ‘university’ in the title anyway.  How?

The use of ‘University’ is prohibited for entities that are doing higher education.  In this way University Technical Colleges (UTCs) are not in breach of the FHEA, although there are criteria for their designation.  University hospitals are not universities, The University Arms Hotel is not a university etc etc.

A potentially confusing development has been the agreement that FECs can call their HE offering a ‘University Centre‘ or sometimes a ‘University Campus’.  There are some criteria for these, but it’s not clear that these are being enforced.

fott

So if UA92 is not a university, a university college, a university centre, or a university technical college, how does it get ‘university’ in its title?  We’re told it’s because of its link to Lancaster University.  Surely that can’t stand?  Lancaster are operating as a partner to UA92 in much the same way that many universities are to their partners.  What’s to stop them applying to DfE to get a ‘non-objection’ to the inclusion of ‘university’ in their company names?   

Maybe DfE could publish their rationale for why UA92 gets to have ‘university’ in its title?  Will it feature a reasoned argument or just a special case for a bunch of former footballers? 

Can you reduce tuition fees by cutting bloated management salaries?

Andrew Adonis is right.  Through all the fun we’ve been having as he’s raised issues about higher education, there have been some key themes he’s returned to.  One is that fees are too high and that you could reduce them by cutting management salaries. As the summer has worn on we’ve had different versions of this formula, and many people have cruelly mocked his lordship’s maths.  Several of the best twitter spats were between Lord Adonis and Danny Blanchflower (Professor of Economics at Dartmouth College) who accused him of getting his sums wrong.

Here’s Lord Adonis interviewed in Cherwell:

However, Adonis argues that those who have challenged him on this point, such as economist Danny Blanchflower, “haven’t done the maths”. At Oxford, if the total pay of the 452 individuals earning over £100k was cut by 25%, every undergraduate could have their fees cut by £1000 next year. A change which would enormously benefit every student, and affect just 3% of its staff.

This is true.  Lord Adonis was right.

Oxford’s published financial statement shows the salary banding of staff above £100k (another of those OfS requirements that universities are already doing). There are 451 people listed there.

oxford

If you add up the 451 people (using an assumption for these purposes that they are all on the midpoint in these salary bands) they are paid a cool £65million.  A quarter of that would be £16.3million, which would easily be enough to reduce home & EU fees by £1000 (Oxford has a little over 10000 home & EU UG students).  There are a few caveats though.  Firstly Lord Adonis doesn’t want this trick to be done just once, he thinks they should reduce fees by £1000 for five years.  That can’t be done just by taking successive 25% slices off management salaries (in the last year they’d all have a negative salary).

Lord Adonis has also continued with his assumption that all 451 are part of a bloated management.   That can’t be right.  For a start, Oxford, as with other universities, splits out clinical staff where it makes payments on behalf of the NHS.  The Medical Sciences division is very large at Oxford – a third of those 451 are getting an NHS payment – that probably constrains the ability to cut those wages.  But, just as the 152 clinical staff highlights that there are doctors in the 451, so a large proportion of the remaining 299 are going to be professors rather than deputy directors of HR.  There’s a clue – the highest paid person in the university got £880,000.  That can only be because they got paid a royalty on some form of intellectual property in that year – why else would they get twice the pay of the VC?  Clearly this person, and many others of the 451, are highly transferable.  After all, Professor Blanchflower has been taunting Lord Adonis from his chair at an Ivy League college in New Hampshire.

So, sadly, if Lord Adonis was right about the maths, he’s hardly right about the data underlying the figures.  This is not a cadre of 451 administrators who’ll gladly take a 25% pay cut.  Even if this could be done, it’s a one-off, a windfall that could not be taken again.

But, although some policy makers and newspapers think that there are only two universities in England, this is not a game that you could easily play at other universities.  This is the version of that table from Oxford Brookes University.

brookes

A quarter of these salaries only raises £337500, which is only 0.4% of Brookes’ £77million Home & EU UG FT tuition fees.  Brookes has slightly more UG students than Oxford, so raiding its management salaries would see a fee reduction of £30 per student.  As an ex-poly, no doubt Lord Adonis has a different fate in mind for Brookes.

Here’s the problem.  ‘Doing the maths’ is superficial.  The assumptions made are staggering, especially those about the uniformity of the sector.  They also refuse to deal with the funding system as it is established.  If Oxford had a windfall £16million it would be well advised to leave its fees exactly where they are. Reducing the fee only really saves the Treasury money – if they had £1000 per student, they’d be much better advised to sink it into supporting students who need the money.

We’re promised a ‘major review’ of student finance and university funding.  It’s vital that it does not start from a naive assumption about costs, from management salaries to differential costs between courses.  Lord Adonis seems to think this Herculean task can be done quickly (he’s referencing the Augean stables now) and he’s wrong again – this won’t take a day or a month, but should take a year.  No more bodged attempts at this please.