I was recently invited to speak at the HE Professional REF Conference in London, which took place on Tuesday 14^th May 2024. As Research Impact Manager and Associate Professor of Knowledge Exchange and Research Impact at Staffordshire University, I naturally opted to focus my talk on ‘Impact and Engagement in REF2029 Submissions: Planning and Preparation’. I wanted to be able to share some of the initiatives that I’ve implemented at Staffs, and to reflect on some of the progress and challenges in terms of developing impact literacy and culture. And of course, given the REF focus of the event, I was also keen to capture the practical REF preparations, and in what ways these might be shaped by the publication of the initial decisions.

Rather than just discussing my own experiences at Staffs, I wanted to offer wider perspectives. To this end, I put a call-out via the ARMA Impact Special Interest Group, the Northern Impact Network and the Midlands Impact Group, asking colleagues to fill in a feedback form. Questions focused on progress, initiatives and challenges for the impact element of REF preparations. I received 27 responses, and really appreciated the time that colleagues had taken to provide often very detailed and thoughtful comments. Whilst not a formal piece of research, I believe that the responses offer quite a broad snapshot of the current state of play for long-term planning for Impact and Engagement in REF2029. In appreciation of all the contributions, I undertook to share the key insights with impact colleagues.

Impact at Staffordshire University

I began my talk by setting the context for my work at Staffordshire University. We are a teaching-intensive and research-inspired institution. We have a strong commitment to the Civic University agenda, which is delivered through our Connected Communities Strategy. This commitment is reflected in our impact case studies, which tend to have a strong focus on public engagement.  We submitted to 7 Units of Assessment (UoAs) in REF2021 and consider ourselves small but mighty when it comes to impactful research, with 87% of our impact being 3* or 4*.

I am the inaugural (and sole) Research Impact Manager and have been in post since March 2018. Since the REF2021 submission, I have been implementing a whole programme of activities and support to enhance impact literacy and culture across the institution, and to address long-term REF planning. The programme includes flexible impact training, some of which is tailored to individual Research Centres. To boost capacity, we now have Impact Leads in the Research Centres, and I offer a ‘train the trainer’ programme of support and development. I developed an initiative called Research Impact Coaching Cohorts (with workshops, one-to one coaching sessions and community building sessions). The cohorts cater for colleagues at different stages of their impact journeys, including a PGR cohort that is currently running. For impact case study authors, there are monthly open workshops. These workshops have a flexible format, so that colleagues can engage in whatever way is most useful to them (e.g. drop-ins; writing retreats; training sessions, and/or peer feedback sessions).  We use our QR Policy Support Funding allocation to support impact acceleration. And we monitor our progress towards REF with annual REF stocktakes (including draft case studies).

Embedding Impact Culture and Literacy: what works?

My own experience aligns strongly with impact colleagues elsewhere, in that preparing for impact and engagement in REF2029 includes a healthy recognition that it’s not just about REF! It’s about embedding impact literacy and improving impact culture, so that impact becomes an integral part of the research process. And aligned with this move is an increasing focus on the ethics of impact, and what constitutes ethical, meaningful engagement. Professional services support is vital. As one response noted:

“Permanent impact support staff with the skills and knowledge to develop a thorough understanding of our research/its impact and our researchers – colleagues who have built trusted relationships and are able to spot opportunities and develop the impact literacy of those researchers in a manner that is attuned to their individual ways of working.”

Impact support is increasingly tailored to individual needs, which has led to a strong interest across the sector in the use of coaching-informed approaches (see our Coaching for Impact webinar series where we are exploring these developments). As well as core professional services staff, some responses mentioned employing additional Research Assistants to support with evidence collection and analysis. Others highlighted the value of external contributions, with consultants providing workshops, consultations, and resources.

As well as Professional Services input, academic leadership was also recognised as a critical factor. There are often now academic impact support roles in Schools/UoAs, and opportunities to share experiential learning and to celebrate success:

“Really useful to include academics in the presentation of successes/challenges/examples of best practice, and for planning to be researcher-led.”

“Plenty of opportunities for informal, but regular, knowledge-sharing, especially where those who led case studies in REF2021 share their experiences with others.”

There was also recognition that senior leadership is vitally important to support both the Professional Services and the academic contributions, providing “an aligned approach and vision that is communicated and supported top-down.” Pump-priming and longer-term funding was also – unsurprisingly – recognised as crucial.

Challenges in Institutions

Alongside the positive picture of more permanent impact support staff, improved impact cultures and greater inclusivity, there were also numerous comments that pointed to significant challenges, both in institutions and across the sector. A key point that came across in the responses is that institutions do not operate on a level playing field. This is perhaps obvious, but nevertheless often overlooked. Whilst some universities have benefitted from many years of Impact Acceleration Funding and have whole teams of staff working together to support impact, others are at a much earlier stage in developing their impact culture. For example:

“As a small institution, this is the first time we have an impact manager. having someone working with departments has made a huge difference, but we also know how behind we are with embedding impact into the research endeavour.”

In addition, the current financial challenges facing universities are having a significant impact in terms of tighter budgets and in many cases, staffing reductions. It was clear from some of the responses that this is seriously affecting staff morale and motivation.

“There have been significant interruptions locally (due to finances / redundancies) which have affected morale.”

“Internal change and transformation, particularly losing institutional experience and knowledge through staff turnover.”

Another response pointed to ‘early onset burnout’, linked to a tension between limited investment (especially in staff) and ambitious targets. In another case, the pressures meant that central support was prioritised towards case studies with the greatest 4* potential. The competing pressures on staff time, especially in teaching intensive institutions, were also raised, and blamed for difficulties in “even getting internal buy in for the importance of impact”. Such challenges will not be experienced equally across the sector. One response called for initiatives to address inequalities:

“We need to quickly develop a sectoral approach to impact support that would reduce the inequality within the HE sector and made access to impact support available where it is most needed. At the moment the universities doing best in attracting funding invest much more into REF preparation which makes the differences in the quality of REF impact case studies across sector even larger.”

A further challenge is that collecting evidence of impact is time-consuming and complex, and there was limited capacity in some cases. It is also more difficult in some disciplines than others:

“It remains SIGNIFICANTLY easier to evidence economic benefit and (some) policy change compared to environmental or health benefit, which is itself easier to gather than societal benefit.”

The same response called for robust tools and support from the Research Councils to help universities to address the more difficult to evidence types of impact.

Changes for REF2029

I asked about whether colleagues were preparing any differently for REF2029 compared to REF2021, including any preparations for the Impact and Engagement explanatory statements, due to be introduced in REF2029. Many responses highlighted improved strategy and clearer timelines, including thinking beyond REF2029 to the longer term. Responses showed how they were starting earlier, routinely collecting evidence, and trying to build on REF2021 success and lessons learned. In some cases, Impact Managers appointed during the previous REF cycle have remained in post, providing on-going support, and implementing processes. Several people talked about greater confidence, for example:

“Hopefully greater optimism in chasing difficult cases that might have seemed too risky last time around.”

Numerous people talked about the value of one-to-one support for case study authors, with the emphasis at this stage being more on planning and recording impacts than asking academics to produce impact narratives. Several responses also referred to an increased focus on internal and external equality, diversity and inclusion.

Alongside the positive developments, there are again some key challenges around the changes for REF2029. An issue causing widespread concern is all the unknowns about Impact and Engagement in REF2029. Perhaps the biggest unknown is what will be required for the Impact and Engagement evidence statements. Given the proposed weighting of the evidence statement according to the size of the UoA/number of impact case studies, this was seen as a particular challenge for smaller institutions. Moreover, for universities that have to collect a lot of the data manually due to having no Current Research Information (or CRIS) system, there are particular anxieties over not knowing what the key performance indicators are going to be for these statements. Other outstanding questions include How will ‘engagement’ be defined? What does ‘rigour’ mean in this context? And how will we evidence it? And there were concerns about the lack of some key timelines; lack of standardised paperwork/permissions for evidence collection, and the implications of processes to determine staff numbers in terms of UoA decisions and size.

Given the uncertainty, how are Impact Officers trying to prepare for Impact and Engagement statements? Are they taking a speculative approach and introducing changes, or are they opting for ‘wait and see’ before changing practice? Again, it varies: some are advocating carrying on as usual:

“We’re not preparing much at present due to not knowing what will be required, not knowing how public engagement will come into this and what other work for the additional statement will be required – at present our message is to keep researching and evidencing any impact that research has!”

Some have done what one person referred to as ‘outline, conceptual planning’. Some are trying to offer academic leads some initial support on planning for the evidence statements. And in one or two cases there was a more systematic approach:

“We have compiled a ‘template’ for guidance, which draws on elements of the REF2021 environment statement and sector knowledge of what is likely to be included this time. We are asking researchers to use it as a guide to thinking about the data they may need to collate, and the activity that might be recorded in it. We anticipate asking them to complete a first draft in autumn 2025.”

Other responses focused on ensuring that governance and other support processes are properly embedded, including consideration of EDI, and supporting opportunities for partnership, collaborative practices, networking and interdisciplinarity. And others just emphasise keeping evidence of engagement and impact activities, and reminding colleagues that it won’t just be impact case studies this time.

I concluded my talk with a big shout-out to all the wonderful impact networks that I belong to and value.  Some of these were absolute lifesavers for me during REF2021 preparations and continue to be a great source of information and mutual support.  This includes the ARMA Impact Special Interest Group; the Midlands Impact Group (that I currently Chair); the Northern Impact Network; Universities Policy Engagement Network (UPEN); Fast Track Impact, and the Research Impact Academy. I’d especially like to thank everyone who responded to my request to share their experiences and insights for this talk. Impact people tend to be the ultimate connectors of people, places and ideas. As such, it’s a wonderful and generous community to be a part of!

Dr Jackie Reynolds, Research Impact Manager

Twitter @DrJReynolds

Linkedin: Jackie Reynolds | LinkedIn

Here at Staffordshire University, we are small but mighty when it comes to doing highly impactful research. Our impact results in REF2021 were the 4^th most improved in the country (compared to REF2014), with 87% of our impact being judged ‘very considerable’ or ‘outstanding’ (3 or 4*). Much of our impact is generated through deep and meaningful co-production with communities in Stoke-on-Trent and beyond. Our case studies (available here) are distinctive in that many have a strong emphasis on public engagement and place-making. There are some clear elements of what Rickards et al (2020) refer to as ‘3^rd Generation Research Impact’ or ‘research impact as ethos’, in terms of enhancing positive impact through building ‘innovation eco-systems’ across projects and other boundaries, focusing on what impacts are most important (especially in terms of place making) and continually working to adapt and improve our contributions as a Civic University.

To further build on these strengths, improving impact literacy (as conceptualised by Bayley and Phipps 2019) more widely is a pressing need. And with some of our case studies being based on the work of individual researchers, there is additional potential for working more collaboratively within the university as well as with external partners. Therefore, as well as getting better at embedding impact into the lifecycle of our projects (e.g. developing compelling impact plans in funding bids, and implementing rigorous impact evaluation strategies), we also need to continue developing our impact culture. My thinking on this has been influenced by Prof. Mark Reed’s book ‘Impact Culture’ , in which he argues that ‘impact culture emerges at the intersection between research, community and purpose’ and that ‘each of the three components of a healthy impact culture are facilitated by sufficient internal capacity…including skills, resources, leadership, strategic and learning capacity’ (Reed 2022:19)

Following our REF submission, I turned my attention fully to addressing these challenges. In 2021, I developed a survey which was distributed to researchers of all stages across the university (PGRs to research leaders). 105 people responded. Over half of respondents (55%) ‘strongly agreed’ and 24% ‘somewhat agreed’ that they were ‘highly motivated’ by making a difference through their research.  However, when I asked respondents to rate how confident they felt (on a scale of 1-10) about developing impact from their research, only 20% rated themselves highly (8 or more out of 10) and almost 30% rated themselves 5 or less out of 10. A further question assessing levels of confidence in evidencing impact also highlighted training needs, with 50% of respondents rating their confidence level as 5 or less out of 10. There was clearly a gap between some people’s ambitions for making a difference and their confidence to achieve and evidence this.  

At the same time, I became increasingly aware that it is not only inexperienced colleagues who require impact training and support. Post-REF, I was also receiving requests for continuing support from case study authors to help them to better embed impact in their project lifecycles, especially in terms of longer-term evaluation strategies. In addition, they wanted to build on their success in REF through on-going funding applications and required support to do this effectively. The complex nature of developing impact from research means that colleagues of all career stages can benefit from on-going training, support and opportunities to share and reflect with others about successes, challenges, and failures too!

Initially, my response was to provide a series of training sessions targeted at our Research Centres and tailored to their specific needs. This was very successful in some instances but varied in terms of levels of engagement. I also initiated a regular ‘Impact Surgery’, with self-bookable 30-minute slots to meet individual colleagues via Teams. Again, this was successful and valued by colleagues. It is continuing to shape my approach. I undertook an Advanced Mentoring qualification and was keen to work more intensively to support individuals, as well as trying to recruit more senior colleagues to also act as mentors for impact. Mentoring is often seen as a positive strategy to build research and impact capacity. However, mentoring is time consuming and requires extensive commitment from senior colleagues. It can often fall to a relatively small group of people to provide mentoring across numerous programmes and areas of work, which is challenging in terms of capacity, and benefits a relatively small number of colleagues.

My ambition was to develop an approach that would be more transformative by benefitting larger numbers of colleagues. Crucially, it also needed to cater for those colleagues being at different stages of their careers and their impact journeys.  Since I am the university’s sole officer for research impact, I also needed to build capacity by including peer support and wider training contributions. Through engaging with wider discussions across the sector, I began to formulate ideas around developing an impact coaching initiative. Prof. Mark Reed suggests that impact can become more of a priority through ‘engag[ing] researchers in a coaching process to identify forms of engagement and impact that they might find intrinsically motivating’…’organis[ing] internal impact-related events that will engage researchers with varying levels of interest and experience with impact’…and ‘harness[ing] the power of your communications in creative new ways’ (Reed 2021 pp.168-174). I was also interested in the example of the University of Leeds, where Dr Ged Hall and colleagues developed a 3 month rolling programme called ‘Building Impact Momentum’ that uses peer to peer support and tutor input to build on the intrinsic motivations of researchers and transform the impact culture of the university.

Inspired by these ideas, I began to plan a new initiative for Staffordshire University: Research Impact Coaching Cohorts. To cater for colleagues at different stages of their impact journeys, there are three cohorts: Impact Essentials, Impact Builders, and Impact Leaders. Impact Essentials is an introductory level cohort for those at the start of their impact journey and might require coaching to envisage their impact and how to implement a pathway to impact. It is primarily (but not exclusively) comprised of early career researchers, including postgraduate researchers. Impact Builders are those who have a track record in achieving impact from research but have not yet contributed to a REF impact case study and/or do not yet lead teams around impact. They may need further skills and support to scale up and/or to evaluate their impact. They will also tend to benefit from a greater understanding of how to develop a REF impact case study. Impact Leaders are those who have contributed to REF impact case studies and/or have other research leadership roles (e.g. Research Centre Leads, UoA Leads in REF2021). They will have good understanding and experience of developing and evidencing impact and will be able to share their experiences with other cohorts. However, they will also have their own developmental needs e.g. evaluating longer term impact, embedding impact in larger funding bids, and developing their impact leadership.

I initially posted a call-out on our Researcher Network Microsoft Teams Community and was delighted by the positive and enthusiastic response. I framed it as a new learning community and scheduled a Launch Event with afternoon tea in our new Catalyst Building in September 2022. A steady stream of researchers continued to join. At the time of writing, around 70 colleagues have signed up across the three cohorts, and I expect this to continue to grow. At the Launch Event, I presented the following aims for the coaching programme:

• To develop understanding, skills and capacity around planning, developing and evidencing the impact of Staffordshire University’s research beyond academia.
• To build a stronger research and impact culture through communication and collaboration across the university
• To support YOU to achieve the impact goals that matter to you most

The first phase of the programme is underway and runs until December 2022. It is designed to be an introduction to different aspects of the impact landscape in HE, along with spaces for sharing experiences, individual and group reflection, and action planning. Activities include:

• Workshops and sharing events to provide an overview of the impact landscape in Higher Education, and linking research, innovation and enterprise e.g. Introduction to Impact; Research Impact and the REF; Communicating for Impact; Insights from Policy Impact Acceleration Funding Awards; UN Sustainable Development Goals; Impact through Knowledge Transfer Partnerships; Creating Impact from Intellectual Property (IP).
• Monthly ‘Net-Walking’ sessions: involving a group walk and discussions in a nearby park, creating physical and mental space for connecting and reflecting on impact. Also designed to promote a sense of community and wellbeing.
• Continuing the bookable 30-minute one-to-one coaching sessions for one morning a week on a regular basis.

In this initial phase, the activities are aimed at all three cohorts, for colleagues to dip into as they wish. The aim is that by Christmas, everyone involved will have at least a basic understanding of the impact agenda and also done some initial reflection on their own impact. I am evaluating each session, with a focus on identifying the most significant insights and what actions participants will take as a result.

The second phase of the 2022-23 programme will begin in January. At this stage, cohort members will be placed in groups, based on the stage of their impact journey and/or the type of impact that they are working towards and/or common stakeholders. There are clearly advantages to mixing the cohorts within the groups so that the Impact Builders and Impact Leaders can share their experiences with colleagues at an earlier stage of their impact journey. The sessions will therefore involve peer support and coaching. They will be driven by specific outcomes for each cohort e.g. identifying impact goals and stakeholders; engaging with those stakeholders; developing an impact evaluation plan, and/or developing a funding bid to support impact activities. Through the group coaching sessions, I will also identify specific training needs and continue to develop bespoke training workshops to address these needs. Again, at this stage, these workshops will be very much outcome and action focused to enable each cohort to progress to specific goals. In addition, I plan to continue the other community-building activities, such as the Net-Walking sessions, if they prove popular.

The biggest challenge that I foresee is one of capacity, especially given the levels of interest that are currently being shown. Similar initiatives elsewhere have tended to include fewer numbers of participants and greater numbers of facilitators! However, I’m deeply motivated and encouraged by the enthusiasm and positive feedback from colleagues. It is clear from the feedback that the scheme is already making a difference to colleagues and changing the way that they think about and approach their impact.

References

Bayley, J. E., & Phipps, D. (2019). Building the concept of research impact literacy, Evidence & Policy, 15(4), 597-606. Retrieved Sep 16, 2022, from https://bristoluniversitypressdigital.com/view/journals/evp/15/4/article-p597.xml

Reed, M.S. (2022). Impact Culture. Fast Track Impact.

Rickards, L. Steele, W. Kokshagina, O. and Moraes, O. (2020) Research Impact as Ethos. Melbourne, RMIT University. Retrieved Sep 16, 2022, from: Research impact as ethos | RMIT Next