Not lamentable, but patchy

This was an exciting week for those interested in the Teaching Excellence Framework (TEF).  Not as exciting as the Green Paper launch, but a week in which officials and ministers were out and about explaining what they mean.  There are many excellent pieces being written about the TEF, but I want to come back to why we need it.

Jo Johnson had explained to UUK why we need a TEF.  He said:

… there is lamentable teaching that must be driven out of our system.  (Johnson 2015a)

The good news is that the BIS Select Committee pressed the minister about the lamentable tag.

Chair: I would be very keen for you to give evidence to us, but just to push you on this, “lamentable” is an extraordinarily strong word. Would you use it again?

Joseph Johnson: I think there are patches of poor-quality provision and whether or not we want to use that word—

Chair: Lamentable patches?

Joseph Johnson: Whether we want to use that word, it certainly made a point. It highlighted the point I was trying to make. I do not see the need to repeat it ad nauseam, but I think I made my point.

So, a point was made, but evidence not produced.


Of course, since September, there has been a cry for the evidence for the lamentable statement, and as BIS went out to engage the sector, they had prepared some areas of evidence – which Johnson and his officials have used this week.  Here they are:


Let us, first of all, look at patchiness in terms of how students themselves see the system.  In the NSS 2015 survey, two thirds of providers are performing well below their peers on at least one aspect of the student experience; and 44% of providers are performing well below their peers on at least one aspect of the teaching, assessment and feedback part of the student experience. (Johnson 2015b)


A third of graduates paying higher fees in England believe their course represents very poor value for money (HEPI academic experience survey 2015) (Payne 2015)


We can also see some evidence of patchiness in terms of outcomes. Dropout rates vary hugely across the system, and you heard earlier from the Director of the Office for Fair Access about the differential attainment rates and progression rates of different groups through university that cannot be explained by prior educational attainment. These dropout rates vary hugely, as I said. Non‑continuation rates for certain black entrants into university are much higher than for white students, for example, and these are differences that cannot be explained by prior educational attainment. They have to do with issues like the degree of teaching support and academic support that students are getting at university. (Johnson 2015b)


54% of Employers think that graduates are not prepared for the workforce (BCC 2014) (Payne 2015)

At least 20% of graduates are not finding jobs in high‑skilled graduate employment three and a half years after graduation, according to the Higher Education Statistics Agency statistics, and it might be as much as 58% according to a survey on this subject by the Chartered Institute of Personnel and Development.  Therefore, there is a very significant body—somewhere between 20% and 58%—of students who are not finishing up in graduate jobs or going on to further study three and a half years after graduation. (Johnson 2015b)


Let us take another aspect of how students perceive their experience.  The Higher Education Academy’s recent survey came out earlier this year, and one third of undergraduates paying tuition fees in England believe their course represents very poor or poor value for money.  … Therefore, from the student perspective, there is certainly some room for improvement. (Johnson 2015b)

21% of students found the information given to them before they started their course vague, 10% found it misleading (Payne 2015)

Now.  There’s a raft of issues with many of these sources of data.  There are problems about collection of data, interpretation of data, and inferences about causation drawn from the data.  Just as one example, the CIPD report is full of problems – and was soon contradicted by far better reports on graduate outcomes.

But.  Just look at the evidence base.  It’s striking that we have no evidence about the excellence of teaching.  We have proxy concerns about the excellence of teaching.  So, to match this, we’ll have proxy measures of the excellence of teaching.


Johnson J 2015a Higher education: fulfilling our potential Lecture at UUK Conference 9 September

Johnson J 2015b Evidence, BIS Select Committee 8 December

Payne P 2015 HE Green Paper Fulfilling our potential Presentation at UUK SE Conference 9 December


3 thoughts on “Not lamentable, but patchy

  1. I think I’d go further, Mike, and say that much of this is not about teaching at all, not even proxy measures. Things like prior information about the course and many aspects of “the student experience” are largely out of the hands of those who do the teaching.

    • Yes – I think ‘teaching’ here stands for everything. I was wondering if ‘Course Excellence Framework’ was more the right description (but that begs the question as to how any of this would be dis-aggregated down to a course or subject level.

      • I personally think that TEF is a non-starter. How can it possibly be operationalised? It can’t be done as the REF is done: you’d need an army of assessors to go to each university to assess each course, and who is going to volunteer to do that? The alternative is to use metrics like NSS scores, but we know those have their limitations too.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s