The Festival of Hidden REF 2023
The first Festival took place in 2023. Our whitepaper summarises the discussions that took place at the Festival. It summarises many of the campaigning themes for the Hidden REF and our community, defines non-traditional outputs, and describes the evolution of the Hidden REF competition. A video of the Festival is available below, you can also hear about the experiences of some of the guests and committee members in a special edition of the Code for Thought podcast, which was recorded at the Festival. You can also take a look through the Festival photos and review the slides from the event, which are linked in the programme below.
| Time | |
| 0900-0930 | Registration |
| 0930-0945 | Welcome and introduction to the Festival (Slides) |
| 0945-1015 | Lightning talks, Session 1 * Steven Hill, Research England (no slides) * Kelly Vere, TALENT & The Technician Commitment (slides) * Emma Karoune, Alan Turing Institute (slides) * Lizzie Gadd, Loughborough University (video) |
| 1015-1045 | Coffee break |
| 1045-1200 | Guerrilla Groups, Session 1: * Roles: Hidden Roles: Perspectives, Paths, and Lived Experiences (Studio 2) * Categories: HiddenREF categories 2020 and 2024 (Board Room) * Evaluation 1: How do we evaluate non-traditional outputs? (Event Suite) * Plus… sticker guerrillas |
| 1200-1230 | Feedback from Guerrilla Group, Session 1 (Event suite) |
| 1230-1240 | Group photo – outside M-Shed entrance (weather dependant) |
| 1240-1330 | Lunch |
| 1330-1400 | Lightning talks, Session 2 * Julie Bayley, University of Lincoln (slides) * Pen-Yuan Hsing, University of Bristol (slides) * Cara Rodway, British Library (slides) * Katie Osgood, University of Portsmouth (slides) * Hollydawn Murray, Health Data Research UK (video) |
| 1400-1515 | Guerrilla Groups, session 2: * Communities: How do we engage with communities to get them to submit non-traditional outputs (Event suite) * Competition: Preparing for HiddenREF 2024 (Board room) * Evaluation 2: How do we evaluate non-traditional outputs? Towards Panel working methods (Studio 2) * Plus… sticker guerrillas |
| 1515-1545 | Coffee |
| 1545-1645 | Panel: discussion around the subjects raised during the Festival (Event suite) |
| 1645-1700 | Wrap up, thank you and future plans |
Details on Guerrilla Groups, session 1
Hidden Roles: Perspectives, Paths, and Lived Experiences
What were the trends within Hidden REF 2020 Hidden role category? What research roles exist that were overlooked by the HiddenREF and REF 2028? How can these Hidden roles be celebrated as categories for Hidden REF 2024?
HiddenREF categories 2020 and 2024
How representative were the Hidden REF 2020 categories? How can the Hidden REF 2020 categories be altered/combined? What other categories would you like to be seen for Hidden REF 2024?
How do we evaluate non-traditional outputs?
What aspects of excellence or esteem are shared by all non-traditional outputs? Is peer review an approriate tool for evaluating non-traditional outputs? What changes need to be made to peer review to allow fairer/transparent and competitive evaluation of non-traditional outputs?
Sticker guerrillas
At any time during the day, you can use the sticker boards to make suggestions that will guide our campaign. You can let us know what you think about the research assessment, suggest changes to our campaign, help us identify new groups to work with – anything that will help further our goals.
Details on Guerrilla Groups, session 2
How do we engage with communities to get them to submit non-traditional outputs
What criteria should we use to evaluate non-traditional outputs? Any alternatives to “significance”, and “Hidden-ness: used in the 2020 competition? What models do you know of that might be of use? Looking at feedback from the morning sessions, what alterations or differences in aspects of excellence and esteem are needed to value and evaluate non-traditional research outputs in these categories? What use are calibration methods for non-traditional outputs? How would you like to see calibration methods run, if at all?
Preparing for the HiddenREF 2024 competition
How do we convince HEIs and members of the research community to submit outputs to HiddenREF2024 and non-traditional outputs to REF2028? What roles are required to lobby HEIs to submit non-traditional outputs to REF2028? How can we achieve this together? How can we help communities to work together to lobby their organisations to value non-traditional outputs
How do we evaluate non-traditional outputs? Towards Panel working methods.
What is the reaction of the morning panels regarding changes to categories? Anything missed? If so, what characteristics of evaluators are preferable? How do we recruit them? How can we promote HiddenREF and convince members of the community to make submissions to the 2024 competition?
Sticker guerrillas
At any time during the day, you can use the sticker boards to make suggestions that will guide our campaign. You can let us know what you think about the research assessment, suggest changes to our campaign, help us identify new groups to work with – anything that will help further our goals.