An amusing, but also revealing, little piece in the Guardian. Which of the great philosophers would have been REF-able? Turns out not that many. The publishing industry didn’t have as firm a grip on our sense of research outputs as they do now. Even more recent figures, though, might have struggled: Heidegger (most of whose work were lecture notes not published until much later), or Wittgenstein (published at the rate of a book every two or three decades).
Category Archives: REF 2014
Interesting data, poor inferences
Apparently, the UK higher education system is a ‘knock-out’ performer. Although the UK system is ranked 8th in the world — not bad at all — what is holding it back on that measure is central investment. Per capita, that is, higher education is not well funded in the UK, and the system as a whole pays the price. However, we can compensate for that, correcting for the relative poverty of the country (compared to, say, the US or Switzerland) and thus its relatively low level of public investment. This yields a different ranking, with the UK way up in second place. (Behind Serbia, interestingly.)
All very interesting. It makes you wonder from where all the calls for the UK system to be over-hauled in one way or another are coming (how marking and degrees are scaled being an old chestnut). If it ain’t broke, don’t fix it. On the other hand, is also makes you wonder what would happen if there was some higher level of investment…
The THE article first cited though, goes one step further. The reason why the UK system punches above its weight in this way is the Russell group — i.e. the concentration of funding into roughly 15% of the institutions. (We should also include a few other Universities who are not technically in the RG, but who enjoy big funding and strong research reputations, such as Leicester or Durham.) The article, or rather the chap being quoted, cites the recent REF results as evidence. Evidence for what, exactly? That research is concentrated, to be sure; that funding is concentrated, definitely. But it is certainly not evidence that concentration delivers ‘bang for the buck’. On the contrary, the fact that an army of researchers, working at the other 85% of institutions, are still doing international quality research with bread-crumbs of funding, much less infrastructure, and no endowments to speak of… THAT is bang for the buck. The cost per international-standard research publication at a ‘new’ university is a fraction that of the Russell Group, and I can prove it.
For example, let’s take two random Universities: Newcastle University and Northumbria. Now, Newcastle did pretty well in REF2014, staying in the top twenty nation wide. They submitted a total of 2806 outputs rated 3 and 4*. Northumbria are quite a way down the REF table, but comfortably in the top 100. They submitted 823 outputs at that quality rating. Now, clearly, there is a concentration of research in Newcastle, who produced three and a half times as many quality outputs. However, Newcastle received 35.6 million pounds in recurrent research funding in 2009-10, while Northumbria received … 3.26 million. In other words, with ten times the funding, they only managed three and a half times the number of quality outputs. It’s the same story when we compare Teeside and, say, York. The former required £6440 to produce an international quality paper, while York required £11,900.
Now, I know this is not the whole story. There is a difference between 3 and 4*, and also we may be comparing inherently expensive forms of research (laboratory sciences) with cheaper ones. On the other hand, in the above calculations, I haven’t included other big sources of funding, which are also highly concentrated, such as Research Council grants, European funds, and so forth. In the absence of more study, the evidence clearly goes against the ‘funding concentration’ hypothesis. (Hold on, who wrote this report that the THE is cheering about? — Universitas21 — and who are their UK members: all Russell Group. I’m seeing a pattern emerging.)
Book review – Advice for new Faculty Members – Robert Boice
Boice’s book was published in 2000, but remains as relevant as ever. It is well known in the United States where some universities used to issue it to all new staff. The book is divided into three sections:
- Moderate work at teaching.
- Mindful ways of writing i.e. how to produce research papers.
- Socialising in your new university.
I am going to focus on the second of these which relates to research and writing of papers.
Boice is a psychologist and was very heavily involved in the induction of staff and follow up processes at his University. This gave him plenty of opportunity to run different experiments and approaches with new staff and to see what worked. He was taking about mindfulness and moderation even back then and despite my strong initial doubts whilst reading the chapters his results are staggering. Ever wondered if you have been doing things wrong? Or rather is there a better way? – well this should make you think.
Boice looked at the exemplars in his study group and compared them against the other staff. Essentially Boice found that successful academics:
- Wrote daily even for a short period (often up to 30 minutes maximum).
- Knew when to stop.
These sessions he terms brief daily sessions (BDS) and he contrasts them with binge writing and also discusses the feelings and emotions associated with both approaches. Chapters 10 and 11 give detailed practical guidance on how to get your writing going (and the results at the end of those two chapters will make you think this is at least worth trying). Chapter 14 also deals with letting go of negative thoughts which is useful especially for perfectionist type academics.
What effect has it had on me? Well I am going to try this approach for the next paper I am writing, it is never too late to try a something new .
If you haven’t come across Boice before then I strongly suggest you have a read, especially if you are a new member of staff, or new to research and writing.
Three copies are now in the Staffordshire University library and you can find excerpts online.
How to win friends and misuse statistics
The 2016 version of one of the big league tables is out, accompanied by a commentary. The commentary is all about how the huge rise in tuition fees several years ago resulted in an equally huge increase in services University’s offer. (Actually, this may have had more to do with the fact that bond yields were historically low; so Universities could borrow more cheaply.)
One piece of evidence for this was the drop in SSR — student to staff ratio — over the period. Great. Except… this figure is not by any means equivalent to ‘class sizes’, as the commentary claims. Indeed, the claim is made that investment in advance of the Research Excellence Framework contributed to lowering this statistic. RAE investment is not primarily investment in teachers. Instead, it is likely investment in senior researchers, whom undergraduates will never see; and in part-time or zero-hours teachers who soak up the load so that headline staff can work on that research project. The number of academic bodies around the place rises, relative to student numbers, but class sizes may in fact go up, as students are herded into big rooms to be taught by a handful of faces whose photographs were not in the prospectus.
And by the way, this is another of the league tables that includes entry points as a measure of quality. People across the University world have been complaining about this for years, but one more time: Imagine, for a moment, that you are reading a movie review. The review says that the plot is incomprehensible and silly, the acting wooden, and the direction lazy. But, it concludes, the film is popular among cinema-goers (who have never seen the movie), and therefore it must be good. Five stars! Movie of the year!
Are Universities efficient with their funding?
Interesting poll reported in the THE: Among Members of Parliament, UK Universities are given a poor rating in the ‘value for money’ / ‘efficiency’ category. This no doubt reflects the political hot potato that tuition fees still are, especially with the prospect of them increasing further. No self-regarding politician, a few months prior to an election, wants to suggest that Universities are doing the best they can with the fee income. And no doubt there is waste in the system, although probably less than many think.
Also reflected in these results is that the decision is now close on how to distribute — and indeed also reform — research funding in the light of REF2014. Reality is more clear here. Even the United States, which has a history of concentrated funding, is moving away from this. Any one concerned about ‘value for money’ in research funding need only glance at the REF results. Take Staffordshire University for example. With research funding amounting to a rounding error on that received by some Universities, internationally excellent and world-leading work is being achieved. A denser concentration of research funding into the hands of fewer Universities is certinly not going to improve the ‘bang for the buck’.
This THE article shows a remarkable consistency in REF submissions that contain exactly 14 or 24 full time staff. Why should this be? Because, if one submits more than 14 full time staff, then one also has to submit three (rather than two) impact case studies. Having a poor impact case study in one’s submission is much more damaging than not having one (even two or three) members of staff included — regardless of how good their research is. Not being included in a REF submission is a potentially career-damaging event. Even those who unequivocally agree that research should have an impact, and that impact should be measured, should find depressing the unintended effects of the REF rules.
Research Excellence Framework funding – a fair share needed
Universities are vital to a successful local and regional economy; many of them are carrying out applied research with business and other stakeholders such as Local Enterprise Partnerships. The Research Excellence Framework funding allocation acts as seed money for many of these universities. For example, the money we have received last time at Staffordshire University has allowed us to lever in EU project funds and to provide staff time to bid successfully for work with companies.
December 2014 provided the announcement of the Research Excellence Framework results for British Universities. What these results demonstrated is that world class research was occurring throughout most of the University sector although often in pockets within a University. The summary for Staffordshire University is here
Research funding in the UK is overly concentrated in two ways:
a. The vast bulk of the money goes to a very small number of Universities, in fact University research funding is the most concentrated research funding in the world. For example, in 2013-2014 the top 9 universities received 51% of research funding, and 87 universities shared just 10% of the funding.
b. An incredible spatial concentration of funding to London and Oxbridge. A further concentration of funding is only likely to exacerbate regional inequalities in the UK.
Decisions on the split for funding will focus on how much is awarded to 3* and how much to 4* units over the next couple of months. We need to see that research rated at 3* receives at least 40% of the allocated budget with the remaining 60% going towards 4* rated research that way world class research will get funded regardless of where it takes place.
Dermot Lynott provides an initial look at the Psychology ref and whether we are getting bang for the buck with some of the most prestigious universities.
Dorothy Bishop has an entry on how the REF exercises over time have led to a divergence in the sector and whether this is desirable.
A wider spread of funding will be best for the UK and best for the regions the Universities serve.