A framework for assessing excellent teaching

Imagine that you are in government, a conservative government.  You want to increase participation in higher education but you want to maintain quality.  ‘Challenger institutions’ will gladly increase their student numbers, but you’d like to incentivise better teaching for the whole sector through making funding linked to better quality.  In 2015 you’d get the department to come up with a TEF. In the late 1980s, you’d have to leave the job to your funding councils…

The 1988 Education Reform Act fundamentally changed the landscape of higher education funding and regulation. For now, the binary line was retained.  For universities the UGC was replaced by the UFC (an act that many predicted was the end of the buffer between government and universities). The act incorporated the polytechnics and colleges – removing them from local authority control – and a new funding body, PCFC, was created.  Looking back, many have noted that the provisions of the 1988 act were a more important moment for the polytechnics than the Further and Higher Education Act 1992 that did away with the binary line.

PCFC were told by Kenneth Baker to look at quality of teaching in allocating funds.  He wrote to them:

“I look to the Council to develop further indicators of both the quality and quantity of institutions’ teaching and should be grateful if it would … consider how these might be used as an input to its funding policies and decisions”
(quoted in PCFC 1990 p7)

PCFC set up a committee of enquiry chaired by Baroness Warnock, then Mistress at Girton College, with the following terms of reference:

(a) to identify characteristics of effective and efficient teaching;
(b) to identify which of these characteristics can be developed as indicators of teaching quality;
(c) to suggest the means by which institutions could demonstrate the effectiveness and efficiency of teaching and the promotion of students’ learning;
(d) to advise on possible strategies for the Council which would serve to raise the quality of teaching. These might include procedures for monitoring and evaluating outcomes, and funding mechanisms.
(PCFC 1990 p7)

FullSizeRender (18)

The committee’s report is short and clear.  Although accused of being too philosophical, it manages to include clear statements about teaching, how it should be taken seriously in the institution, issues for students and employers and then how to demonstrate quality.

The committee saw their principles in a framework:

(a) that ‘teaching’ must be interpreted broadly;
(b) that teaching must be responsive to students’ needs;
(c) that the conditions necessary for good teaching must be taken seriously, their satisfaction given priority at every level of institutional activity.
(PCFC 1990 p11)

The conditions  are spelt out in a further framework regarding curriculum, staff development, student and employer engagement and self-evaluation.  The committee notes that the PCFC ‘if seeking to identify excellence in teaching, must look above all for evidence of commitment to such  excellence, not just on paper, but throughout the management and practices of the institution’ (PCFC p13).

The committee’s advice is given in broad terms, recommending six strategies that PCFC might use.  These included setting minimum conditions for teaching quality; teaching enhancement; self-evaluation; monitoring of teaching strategies.  The committee also suggested that PCFC might look at how excellence might be related to an institution’s ethos.  One strategy is worth looking at further; the committee suggested the promotion of quantifiable and other information by which progress and achievement can be demonstrated.

Quantifiable Information

The late 1980s were a time of Performance Indicators.  The committee drew on both existing and developing work in framing its strategy. Institutions would have their own criteria to measure the quality of its teaching.  The committee suggested that PCFC ‘will wish to use comparative national data both between institutions and over time to make possible realistic and well-founded comparisons’.

The committee supported the use of data on:

student progress, satisfaction and subsequent employment; expenditure on staff development; the cost of a course compared to the degree results obtained; and completion rates
(PCFC 1990 p29)

25 Years later, we are unlikely to see expenditure on staff development in the TEF, but the other indicators?.  The committee also looked forward to further work on that PCFC was doing on measuring value-added…

The report shows the quality you’d expect from the committee’s exalted membership and the 22 months they took to complete it.  Much of the report’s work underpinned the quality assurance system that followed the abolition of the binary divide, but the direct link between quality and teaching funding never came (as opposed to funding streams such as FDTL or special projects such as CETLs).   There’s much to commend to contemporary policy makers, especially those thinking about TEF, in this 25 year old report.

 

Reference
PCFC (1990) Teaching Quality – Report of the Committee of Enquiry Appointed  by the Council,

Advertisements

13th Century Higher Education Policies

It is complicated to compare current policy issues with the way that they’ve been handled in the past; after all the conditions are never the same.  But it is illuminating that some issues have remained constant through history, even if the solutions have varied.

Henry 111

During the long reign of Henry III the English universities moved into a new relationship with the king – cementing a role as the king’s universities.  They gained a series of privileges from the king and he acted as their protector.  A key example is how the interests of Oxford led to the king ordering the closure of a university at Northampton.  Imagine that; the interests of Oxford being prioritised over newer universities…

‘If the university … persisted there, it would much harm our town of Oxford … especially as all the bishops of our land have signified by their letters patent that the university should be moved from the town for the utility of the English church and the advancement of students’
Quoted in Lawrence (1984)

Pearl Kibre (1960) highlights other parts of what we might, anachronistically,  call Henry’s ‘university strategy’.

Consumer Protection

Henry III saw himself as the guardian of the scholars and confirmed many of their privileges particularly by moving authority for matters that concerned them from local control to the university chancellor.  The chancellor was given jurisdiction over contracts between scholars and others.   The university also gained increasing power over the provision of services to scholars.

Henry III granted … the right to be present … at the trial of bread and ale. … The examination of the bread was to be made twice each year … The assize of the bread was to be made at the same time and was to be based on the price of grain.

… in 1285, in accordance with the grant of Henry III, Edward I provided further that all bread and ale which did not meet the specifications of the assize were to be declared forfeited to the king.
(Kibre (1960) pp283-284

International Strategy

Henry III supported the recruitment of international students – helping to recruit them, house them and provide a 13th century version of post-study employment…

… in 1229, he invited the disgruntled scholars of Paris to migrate to England during their dispersion, and offered … to assign them cities, towns, or villages if they would transfer to his kingdom, for purposes of study.  King Henry III shortly afterwards in 1231, followed up this decree or invitation with another pronouncement on the scholar’s behalf. He did so, he declared, in response to the complaints about rents brought to him by members of the university. Henry III at this time called attention to the fact that the entire kingdom was honoured by the presence at Oxford of scholars from all parts of England as well as from lands across the sea.
Kibre, 1960, p270

Henry set about confirming a form of rent control and his son, Edward I, confirmed the allocation of housing to students.

Fraudulent students

However, these rights given to scholars had to be jealously  guarded.  It had been possible for scholars to attach themselves to the university, but these new rights meant that the university’s authority over them had to be confirmed.

From an early period, at Oxford, an effort was made to prevent the growth of abuses associated at Paris with the difficulty of distinguishing between legitimate scholars and those fraudulently claiming this status. To achieve this objective Henry III endeavoured to promote  a close cooperation between the sheriff of Oxford and the chancellor of the university.  In in 1231 he directed that that the sheriff expel all so-called fictitious scholars from the city and that he make certain that no clerk who was not under the direction of a master of the schools remained in Oxford.
Kibre, 1960, p 271

Kibre notes that at Cambridge, the sheriff offered the chancellor free use of the city gaol for fraudulent students.

Right from the very outset of universities it is clear that were evolving a relationship with the state in which protection was linked their role as the king’s universities.  It’s also clear that 750 years later, many of the same issues remain with us – even if universities no longer control the price of beer.

References

Kibre, P, 1960, Scholarly Privileges in the Middle Ages, London, Mediaeval Academy of America
Lawrence, C, 1984 . ‘The University in State and Church’ in Catto J (ed) The History of the University of Oxford Vol 1, Oxford, Oxford University Press

Not lamentable, but patchy

This was an exciting week for those interested in the Teaching Excellence Framework (TEF).  Not as exciting as the Green Paper launch, but a week in which officials and ministers were out and about explaining what they mean.  There are many excellent pieces being written about the TEF, but I want to come back to why we need it.

Jo Johnson had explained to UUK why we need a TEF.  He said:

… there is lamentable teaching that must be driven out of our system.  (Johnson 2015a)

The good news is that the BIS Select Committee pressed the minister about the lamentable tag.

Chair: I would be very keen for you to give evidence to us, but just to push you on this, “lamentable” is an extraordinarily strong word. Would you use it again?

Joseph Johnson: I think there are patches of poor-quality provision and whether or not we want to use that word—

Chair: Lamentable patches?

Joseph Johnson: Whether we want to use that word, it certainly made a point. It highlighted the point I was trying to make. I do not see the need to repeat it ad nauseam, but I think I made my point.

So, a point was made, but evidence not produced.

johnson

Of course, since September, there has been a cry for the evidence for the lamentable statement, and as BIS went out to engage the sector, they had prepared some areas of evidence – which Johnson and his officials have used this week.  Here they are:

Satisfaction

Let us, first of all, look at patchiness in terms of how students themselves see the system.  In the NSS 2015 survey, two thirds of providers are performing well below their peers on at least one aspect of the student experience; and 44% of providers are performing well below their peers on at least one aspect of the teaching, assessment and feedback part of the student experience. (Johnson 2015b)

VFM

A third of graduates paying higher fees in England believe their course represents very poor value for money (HEPI academic experience survey 2015) (Payne 2015)

Outcomes

We can also see some evidence of patchiness in terms of outcomes. Dropout rates vary hugely across the system, and you heard earlier from the Director of the Office for Fair Access about the differential attainment rates and progression rates of different groups through university that cannot be explained by prior educational attainment. These dropout rates vary hugely, as I said. Non‑continuation rates for certain black entrants into university are much higher than for white students, for example, and these are differences that cannot be explained by prior educational attainment. They have to do with issues like the degree of teaching support and academic support that students are getting at university. (Johnson 2015b)

Employablity

54% of Employers think that graduates are not prepared for the workforce (BCC 2014) (Payne 2015)

At least 20% of graduates are not finding jobs in high‑skilled graduate employment three and a half years after graduation, according to the Higher Education Statistics Agency statistics, and it might be as much as 58% according to a survey on this subject by the Chartered Institute of Personnel and Development.  Therefore, there is a very significant body—somewhere between 20% and 58%—of students who are not finishing up in graduate jobs or going on to further study three and a half years after graduation. (Johnson 2015b)

Information

Let us take another aspect of how students perceive their experience.  The Higher Education Academy’s recent survey came out earlier this year, and one third of undergraduates paying tuition fees in England believe their course represents very poor or poor value for money.  … Therefore, from the student perspective, there is certainly some room for improvement. (Johnson 2015b)

21% of students found the information given to them before they started their course vague, 10% found it misleading (Payne 2015)

Now.  There’s a raft of issues with many of these sources of data.  There are problems about collection of data, interpretation of data, and inferences about causation drawn from the data.  Just as one example, the CIPD report is full of problems – and was soon contradicted by far better reports on graduate outcomes.

But.  Just look at the evidence base.  It’s striking that we have no evidence about the excellence of teaching.  We have proxy concerns about the excellence of teaching.  So, to match this, we’ll have proxy measures of the excellence of teaching.

Refs

Johnson J 2015a Higher education: fulfilling our potential Lecture at UUK Conference 9 September

Johnson J 2015b Evidence, BIS Select Committee 8 December

Payne P 2015 HE Green Paper Fulfilling our potential Presentation at UUK SE Conference 9 December