Imagine a REF that recognises the contribution of everyone involved in the development and realisation of research – the technicians, the Research Managers, the non-academic gatherers of data? You may not have heard of the Hidden REF, but this small group, which started in 2020, has influenced the quite revolutionary change to REF 2028 rules that will now allow all staff involved in research to be recognised as part of the submission, not just academics.
Universities shouldn’t have to ask what a good research culture looks like, says Gemma Derrick.
The UK university sector is exhausted. We are suffering a hangover from Covid-19 and industrial action. There is a rising murmur that researchers and their institutions are bruised and now is not the time for change.
On top of this come the rule changes proposed for the 2028 Research Excellence Framework—in particular, the plan to make assessment of people, culture and environment worth 25 per cent of the total. This would replace the environment statement that was worth 15 per cent in previous exercises.
Faced with such a high-stakes change, worries and quibbles are understandable. Some universities have pointed to uncertainties about how research culture can be defined and assessed, and expressed doubts that it can be. Some are lobbying to return to the definitions and weightings used in previous REFs.
But the reception for REF 2028 also shows precisely why the change is needed. Calls to maintain the status quo highlight universities’ inability to see past their desire to compete with one another and think instead about what is best for their own research communities.
Peter Schmidt from the Code for Thought podcast visited the Hidden REF and recorded a special episode about the Festival and our work. In the podcast you can hear from some of our guests and some of the committee members who talk about the Hidden REF and its goals.
How can the Festival of Hidden REF influence REF 2028?
The research landscape is changing, yet current research assessment practices have failed to keep up. The way in which research is judged overlooks many of the people who are vital to its success. We must find a way to celebrate non-traditional research outputs, the people, and practices contributing to research, and this is exactly what the Festival of Hidden REF has set out to do. In this latest blog, Steven Hill, Director of Research at Research England, offers their view on the role of the Festival of Hidden REF and the potential to influence the next Research Excellence Framework (REF) in 2028.
Author: Penny Trees (Emerald Publishing) on behalf of Hidden REF Committee
The need to rethink what and who should be recognised and rewarded in the next REF, the UK’s system for assessing the excellence of research in UK higher education providers, has already been acknowledged by those bodies involved in designing the next research assessment exercise. Research England are one of those bodies and as their Director of Research, Steven Hill will be in a unique position to offer further insight on the changes being addressed. In the lead up to the Festival of Hidden REF, Steven has offered just some of his thoughts on what more needs to be done.
In terms of the actions UK Higher Education Institutions (HEIs) can take to help shape a more diverse and representative set of submissions throughout the next REF cycle, Steven believes choosing which research outputs to focus on should be a key part of research and impact strategy at all levels, from HEI strategy to individuals’ choices. The key question ought to be, “What should I do to best advance knowledge and/or lead to societal impact?”. Steven believes in the need for incentive structures to be aligned with this, so incentives reward the best outcomes, and not necessarily focus on specific output types that are perceived to be more prestigious. The next REF is committed to doing what it can in this direction, says Steven, but HEIs should also be looking at their internal incentives, such as hiring and promotion practices, to encourage alignment.
When discussing the main barriers to HEIs submitting more non-traditional outputs, Steven puts this down to a lack of confidence that non-traditional outputs will be assessed fairly by REF assessment panels. “I don’t think it is the case that panel members have a hierarchy of outputs. They make their very best efforts to evaluate everything they receive fairly. But there are real challenges and barriers that we need to address.”
A challenge noted by Steven is making sure non-traditional outputs can present and give some context to the significance and importance of the research. For example, it is not necessarily obvious from a dataset why it is important. Steven suggests making better use of accompanying statements alongside non-traditional outputs as one solution to investigate.
A second challenge, in Steven’s view, is that the impacts and benefits of non-traditional outputs will often be about the extension of knowledge more broadly than just through the research of those that create the tools. For example, software tools can often contribute to a diverse array of research beyond the researchers who created the tool. “This has been addressed in REF 2028 by having a space for showcasing these broader impacts.” says Steven.
Next steps
Steven Hill is one three guest speakers invited to add their expert views and knowledge to the discussions at the Festival of Hidden REF, taking place on 21 September in Bristol, UK. The festival will bring together a community that is fighting for a more effective and fairer system of evaluating success in research. Those attending will collectively contribute to a whitepaper document that will include all ideas and insights gathered from the festival, with the aim of shaping the next Hidden REF 2024, and REF 2028. For more information, visit hidden-ref.org
The new REF rules are welcome, but change will only truly occur if institutions finally get over their obsession with publications, says the Hidden REF committee in a new article for Times Higher Education.
The recent publication of a series of reviews and the early decisions for REF2028, has highlighted an increased focus on research cultures and environments that extend beyond traditional researchers and research publications. The Hidden REF committee argue that submissions should include 5% of non-traditional outputs and present the Hidden REF 5% Manifesto as a starting point to realise this goal.
Cat Davies, Dean for Research Culture at the University of Leeds, explores the hidden consequences of the shift towards assessing research culture in REF2028.
“This win demonstrates the power of careful evidence-based lobbying by courageous, creative groups such as the Hidden REF(‘Celebrating all research outputs’) who should now take their moment of glory.”
Bringing Non-Traditional Research Roles in from the Cold
The Festival of the Hidden REF
The inaugural Festival of Hidden REF is coming to Bristol on Thursday 21 September 2023. This unique event will bring together policymakers, publishers, and people in non-traditional research roles to explore ways of improving the assessment of research in the UK. You can sign up to be notified about early registration on the Festival website. General registration is free and will open on 3 July 2023.
Founded in 2021, the Hidden REF is a grassroots campaign aimed at securing recognition for non-traditional research roles and practices that research assessment bodies have historically overlooked.
While the UK’s Research Excellence Framework (REF) is open to examining a diverse range of research outputs and the broad interdisciplinary community of people who create them, institutions tend to be conservative with their submissions. As a result, the REF ends up focusing on publications – and the select few academics who are named. Most other members of the vast and rapidly evolving research community – that is, librarians, Research Software Engineers, technicians, lived experience contributors, and countless other established and emerging disciplines – are absent from the picture of UK research that the REF is constantly building and shaping through its assessments.
This chronic oversight constrains the potential of UK research by perpetuating a regime in which recognition fails to reach the individuals and institutions who often carry out the most innovative research.
Recognising this, the Hidden REF was launched in 2021 as a competition, allowing individuals to submit examples of research that fell outside the criteria normally submitted to the REF. It was a phenomenally valuable exercise, garnering attention from major scientific publications, generating a huge amount of online commentary, and even moving one submission reviewer to tears. Having begun as a one-off event, the Hidden REF quickly became a movement, pressing for vital reform in UK research.
The Festival of Hidden REF is the first opportunity for the members of that movement to gather in one place, exchange ideas, share stories, and collectively plot a more positive way forward for the UK research community.
Taking place at M Shed, an innovative dockside museum space on Bristol’s Princes Wharf, the Festival of Hidden REF is a free one-day event made up of lectures, break-out groups, expert panels, and more. The event will be attended by policymakers from across government and academia, as well as those involved in designing research assessment frameworks in the UK and abroad.
More importantly, it will be attended by those who have contributed to research without receiving the proper recognition – the hidden roles – either for the contributions they have made or for the potentially valuable outputs they have helped to create.
If your work falls into any of the categories set out by the Hidden REF–or, more importantly, an entirely new category that we haven’t thought of yet–we hope to see you in Bristol on 21 September.
The Festival of Hidden REF is an open and collaborative forum that relies on new ideas and perspectives. To that end, a working document will be curated throughout the event, incorporating delegates’ ideas and insights, and eventually disseminated throughout the research community in a bid to bring these ideas into the light.
Whether you can make it or not, we hope to hear from you. Please get in touch with your ideas and questions.
If you would like to help with the organisation and running of the Festival of Hidden REF, meanwhile, we encourage you to sign up.
Keep checking the Hidden REF website and Twitter for the latest news.
Everyone wants to be a Nobel prizewinner, but Nobel prizewinners didn’t get there on their own. Alongside the support of family and friends, scientists rely on an army of technicians, librarians and other people who fill roles that contribute to research outputs — from the humble article to paradigm-shifting experiments.
These essential, hidden roles are rarely celebrated alongside research achievements, which can make it hard to convince graduates in science, technology, engineering and mathematics to consider them as career options. Recognizing the variety of parts in a functioning scientific culture is a challenge that research-evaluation processes around the world have failed to solve.
In 2021, in the shadow of the pandemic, we took part in an exercise that became a live experiment of how hidden contributions can be celebrated…
