Where to now for REF? BISA responds to the Future Research Assessment Programme

In June this year the Future Research Assessment Programme (FRAP) released an initial set of proposals that set out how REF2028 is likely to operate. These initial decisions indicate that REF2028 is going to look very different to REF2021. Over the summer we sent out a survey to our membership, and presented the results to our Directors of Research forum where we gathered further feedback. You can now read our submitted response. For more context and information read on.

For UK-based academics, the Research Excellence Framework (REF) has long played an arguably disproportionate role in shaping careers, individual research directions and the institutional priorities of universities. Even if the aim of REF has been to create an objective measure of the research performance of institutions, in practice, REF looms over academic life in ways that frequently surprise our academic peers based in other parts of the world. 

The last round of the REF saw some significant changes to the process on the back of the 2016 independent review undertaken by Sir Nicholas Stern. Many of these changes were welcomed – the removal of the requirement that individuals each submit four outputs to their unit of assessment’s output pool (in favour of between one to five) alongside the requirement that all research active staff had to be included in the output pool, was felt by many to be a significant improvement on past practice and led, in many cases, to the emergence of far more supportive and less punitive research cultures within institutions and departments. But REF remained an administrative headache and a costly endeavour and so even by the end of the last REF cycle, moves were afoot to think about how to streamline the REF process.

In June this year the Future Research Assessment Programme (FRAP) released an initial set of proposals that set out how REF2028 is likely to operate. These initial decisions indicate that REF2028 is going to look very different to REF2021. One of the most notable changes is that there will no longer be a linkage between individual academics and the set of outputs that form a unit of assessment’s (UoA’s) submission to REF. Alongside this change, FRAP proposes new categories for assessment and different weightings for these categories. Outputs become ‘contribution to knowledge and understanding’, impact becomes ‘impact and engagement’ and environment becomes ‘people culture and environment’. The weightings for these elements have shifted from 60: 25:15 to 50:25:25. At the same time, the FRAP promises a renewed commitment and emphasis on EDI and a focus on efficiency savings via streamlining processes which, it is argued, are likely to decrease REF-related workloads.

The initial FRAP decisions were released by UKRI alongside a call for consultation on some specific elements of the proposals. In drafting our response we launched our own process of consultation – aimed at finding out what our membership thought of these proposals and using this consultation process as a basis for drafting a response to the FRAP consultation. Over the summer we sent out a survey to our membership, in September the results of the survey were presented to BISA’s Directors of Research forum and to the BISA executive committee where we gathered further feedback.

You can now read our response. It is worth noting that even though the FRAP consultation was extremely limited in scope, we have sought to use the consultation process to set out some of the wider concerns that BISA has with the proposed REF changes. Broadly speaking these concerns fall into three main areas (1) the EDI implications of the proposed changes, (2) the unintended consequences of the revised REF rules that are likely to stem from institutions seeking to ‘play the REF game’ and (3) the workload implications and bureaucratic overload that the changes are likely to generate. In short, even though FRAP talks up commitments to EDI and streamlining workloads, our membership was unconvinced that these commitments would play out in practice. As our response shows, there were certain issues that we felt were particularly important for BISA to address.

First, BISA as an organisation is deeply committed to supporting postgraduates and early-career academics – who make up 62% of our membership – and those from traditionally under-represented backgrounds. We are concerned that the EDI talk within the FRAP does not go far enough and that the removal of requirements to declare special circumstances such as disability or caring responsibilities alongside a delinking of individuals from outputs will have negative consequences for many scholars. As some of the respondents to our survey commented:

Second, we have concerns about how the new REF rules may undermine the health of our discipline more broadly. Our consultation with members and research directors revealed very real fears about how the benefits of the new rules would fall to those with institutional wealth; institutions who have been able to invest in initiatives to support impact, engagement and research environment. Additional statements detailing the contribution of unit level research to the discipline and collective engagement endeavours will catalyse a new industry of consultants paid to advise on how to best write them—those with resources will be able to avail of their services while others will not. Unevenness in the breadth and depth of institutional networks inevitably will lead to differential access to information about how to best prepare submissions. Moreover, the potential return of the ‘research superstar’ transfer market with the proposed loosening of rules around connections to units (a minimum of 0.2FTE for 6 months) will enable more financially secure institutions to rent-to-buy 4* research from around the world.

There were of course areas that our membership were more positive about – for example, many respondents to our survey felt quite positive about the greater emphasis on impact and public engagement that is set out in the proposals.

But, by and large, our membership, members of the Directors of Research forum and the BISA executive, see the changes to the REF process as a significant challenge that could have extremely negative consequences for individuals and the discipline of International Studies more broadly. The delinking of individuals from outputs for example, could encourage a culture of investing in ‘research superstars’ over long term investments in early career scholars and those from under-represented backgrounds. It is telling also that almost none of the responses to our survey question ‘Do the proposals, in your view, suggest that the administrative and workload burdens associated with REF will be reduced?’ was answered positively.

Where to now?

Despite BISA’s dissatisfaction with many aspects of the FRAP, we also are aware that many of the decisions have now been made and we are unlikely to see significant changes to the rules. In our response to the consultation, we have therefore sought to constructively engage with the FRAP in order to suggest some ways in which the proposals might be implemented in ways that better meet the needs of a diverse academic community of international studies scholars and departments. This includes:

• We have raised concerns about the EDI impacts of the FRAP and have suggested that there needs to be strict content guidance with clear criteria for all of the narrative elements of a UoA’s submission. This could include, for example, requiring UoAs to include an account of how they populated their output pool, how EDI considerations are reflected in its make-up, and the extent to which teaching only and/or temporary contracts contribute to making the unit’s research environment possible.
• We have voiced our strong opposition to proposals that the work of teaching only staff and PhD students should be included in the output pool.
• We are also opposed to the suggestion in the FRAP that a 0.2 contract held for a minimum of 6 months is enough to demonstrate a substantive link to the submitting institution, a proposal that, in practice will lead to the buying in of so-called ‘research superstars’.
• We have sought to relay the significant concerns of our members relating to the ongoing impact of the COVID-19 pandemic on research and have emphasised the need to account for the pandemic in addressing EDI considerations.

Do have a read of our full response to get a better picture of our current position on the FRAP and REF in general. We will continue to work with our community of academics, including through our Directors of Research forum, to support and to share best practice. Acknowledging that many of our ECR members may be feeling especially confused about the impact of REF changes for them, we will look to run some ECR-oriented events later in the year to discuss and demystify REF. Finally, guided by our membership, we will continue to do what we can to defend their interests and the interests of our discipline.