Submissions to the Hidden REF 2024 competition will be made in the following categories.
Hidden Role
| Title | Hidden Role |
| Category no. | 1 |
| Description | A number of people provide skills that are fundamental to the pursuit of research, but their role is not always recognised in traditional research outputs. The Hidden Role category allows you to recognise the people who contribute towards successful research. We want to recognise all roles that are important to research, these include – but are not limited to – Clinical Trials Managers, Data Stewards; Data Professionals and Data Managers; Librarians, Technicians, Lived Experience Contributors (such as patients),Community-based researchers and other external research partners, Public Engagement Professionals (PEPs), PRISMS (Professional Research Investment and Strategy Managers), Professional Services Personnel, Researcher Developers, Research Knowledge Engineers, Research Software Engineers, Research Technical Professionals, RMAs (Research Managers and Administrators, and probably many other roles that are currently unknown to us (you can name the role in the submission). |
| Assessment | The submission could include the level of support provided, notable times that their skills were fundamental to the achievement of a success, a discussion of their level of skills, any help they have provided to transfer their skills, and any other notable impact from their work. People can submit themselves as a Hidden Role or people can be submitted by someone who works with them. If you are submitting someone else, you must get their permission before you submit them. |
Training materials and courses
| Title | Training materials and courses |
| Category no. | 2 |
| Description | Training materials are vital in ensuring that knowledge is disseminated across the research community. This includes Open Access content. The Hidden REF will recognise the time and dedication invested into the creation of new materials or courses, and significant efforts in updating existing materials and courses. |
| Assessment | A submission should reflect the demand for the materials (e.g. number of people trained) and the utility of the taught skills to research. |
Citizen Science
| Title | Citizen Science |
| Category no. | 3 |
| Example submission | The great garden spider hunt |
| Description | Citizen science is research conducted, in whole or in part, by members of the public.This includes Public and Patient Involvement and Engagement. Citizen science can drive advancements in research, as well as increasing the public’s understanding of and trust in science. Large volunteer networks often allow scientists to accomplish tasks that would be too expensive or time consuming to accomplish through other means. Volunteers in citizen science gain hands-on experience doing actual research. |
| Assessment | Submissions will describe the importance of the research, the method of participant engagement, and the motivation for your approach. Submissions should demonstrate a positive experience for participants, and describe the importance of the data collected and the results generated. |
Research infrastructure
| Title | Research infrastructure |
| Category no. | 4 |
| Description | Research infrastructure covers the full range of everything that is used to power research. From telescopes to biobanks, from museum collections to software repositories, from research ships to seed vaults. Any infrastructure that supports research can be submitted to this category. |
| Assessment | The submission should include a description of the importance of the infrastructure to research, and any information related to the impact that the infrastructure has achieved, such as the number of people it has supported, the research that has been conducted, the support around the infrastructure to help people access it. |
Community Building
| Title | Community Building |
| Category no. | 5 |
| Description | Communities both inside and outside higher education play a hugely important role in supporting researchers and helping to develop the collaborations that are so vitally important in modern research. As research becomes increasingly trans-disciplinary, communities can provide valuable knowledge exchange, skills development opportunities, a stronger voice in contributing to and influencing policy discussions and links to key bodies such as funders and professional societies. This all, ultimately, contributes to the production of higher quality research outputs. |
| Assessment | Success in this category will be determined by factors such as the size and longevity of a community (relative to its intended reach, e.g. local or national), the type of activities it runs and how many people they reach, feedback from members (e.g. via event feedback questionnaires), other support for members (newsletters, technical clinics, etc.), funding won to support the community, training provided by the community, and policy changes as a result of community efforts. |
Grimpact
| Title | Grimpact |
| Category no. | 6 |
| Example submission | The “oops” and unintended consequences of economic modelling for COVID-19 |
| Description | The REF Impact agenda focuses on positive effects, such as benefits and successes. However impact is more complex, unpredictable, contagious, involves multiple partners and many times is beyond the control of the researcher. This award sheds light on research that has had an equal and opposite effect (in terms of significance and reach) on society. |
| Assessment | This category will be judged on a narrative based justification, similar to the current REF Impact criterion. Credit will be attributed to organisations and individuals that have put in place mechanisms to slow the growth of Grimpact, increase responsible research and innovation, and have minimised the potential for negative effects of research. |
Standards
| Title | Standards |
| Category no. | 7 |
| Example submission | Readme Citation Format |
| Description | Standards are used to allow communities and interested parties to work together more easily. If applied too early they can inhibit innovation, if applied too late they can encourage poor practices. They may also be part of regulations or legal frameworks in particular areas such as medical devices. This category includes technical standards for software, data and metadata standards, practice or community standards, and may include other types of standard too. |
| Assessment | This category will be judged on criteria such as if the standard is accepted and used by a particular community, if standardisation allows collaboration or sharing or data or resources and if standards are entered into law or other policy. |
Research teams
| Title | Research teams |
| Category no. | 8 |
| Description | Research projects rely on teams of people and we should consider contributions at every stage of a research project, demonstrating the importance of teams and collaboration, especially where non-research roles, such as professional services and/or support staff, are integral to the development and practice of research. Submitting to this category brings a new approach to the assessment of people, culture and environment. This category is also an opportunity to demonstrate the importance of team science and international collaboration. |
| Assessment | Assessment criteria will be based on the level of contribution the team has achieved and the amount (or lack of) support the team receives. |
Public Engagement
| Title | Public Engagement |
| Category no. | 9 |
| Description | Public engagement helps communities to contribute to, and benefit from, knowledge, teaching and research to realise the value of public and community engagement, for long-term change. |
| Assessment | This category will be assessed on the scale and quality of engagement achieved, the impact of that engagement and the innovation in how the engagement was conducted. |
Campaigns
| Title | Campaigns |
| Category no. | 10 |
| Description | Campaigns raise awareness of issues that need to change to improve the research environment. They can initiate change that is adopted across the research community and creates significant positive impact. They might be local campaigns to change a specific area of the research, regional, national or international campaigns that effect change across a broader area |
| Assessment | Campaigns will be assessed on the scale of the change that is being sought and the progress towards achieving change. They might be judged on the impact or novelty of the techniques used by the campaign, the openness of the campaign to share and support others in the research community and the goals of the campaign itself. |
Everything else!
| Title | Everything else! |
| Category no. | 11 |
| Description | It’s impossible to know everything that’s happening in the diverse research world, so we have the everything else category so that people can raise awareness of a category that we might have missed. |
| Assessment | Submissions to the everything else category might lead to the creation of a new output category in the Hidden REF with its own assessment criteria. |
Physical Artefact
| Title | Physical Artefact |
| Category no. | L |
| Description | Artefacts, objects or craftworks, exhibited, commissioned or otherwise presented or offered in the public domain, for example visual arts, craft and cultural creations. |
| Assessment | This is a non-publication category from the standard REF and, as such, is assessed on the basis of significance, impact and rigour. |
Devices and Products
| Title | Devices and Products |
| Category no. | P |
| Description | An element, system or substance developed to perform a particular function, set, or combination of functions. Incorporates developing the concept and the design and development of any chemical, mechanical, electronic and software components, and where appropriate the overall system architecture. |
| Assessment | This is a non-publication category from the standard REF and, as such, is assessed on the basis of significance, impact and rigour. |
Performance
| Title | Performance |
| Category no. | M |
| Description | A single or series of public events, or short-term, long-term or permanent installations, at which works of interest are displayed. This may take the form of moving image, sonic, visual or other digital media or written text, or a combination of these, as appropriate, to enable the panel to access the research dimensions and/or the researcher’s contribution to the exhibition and to assess its significance, originality and rigour. |
| Assessment | This is a non-publication category from the standard REF and, as such, is assessed on the basis of significance, impact and rigour. |
Patent/published patent application
| Title | Patent/published patent application |
| Category no. | F |
| Description | Granted patents, copyrights, trademarks, or registered designs on specific products or processes. Patents can have been granted in the UK or another patent-awarding country. |
| Assessment | This is a non-publication category from the standard REF and, as such, is assessed on the basis of significance, impact and rigour. |
Composition
| Title | Composition |
| Category no. | J |
| Description | An original published/publicly available score, first performance or first recording by a record label of a musical composition. |
| Assessment | This is a non-publication category from the standard REF and, as such, is assessed on the basis of significance, impact and rigour. |
Design
| Title | Design |
| Category no. | K |
| Description | A creative research/problem-solving output in the form of design drawings, books, models, exhibitions, websites, installations or built works. This may take the form of moving image, sonic, visual or other digital media or written text, or a combination of these, as appropriate, to enable the panel to access the research dimensions and/or the researcher’s contribution to the design and to assess its significance, originality and rigour. |
| Assessment | This is a non-publication category from the standard REF and, as such, is assessed on the basis of significance, impact and rigour. |
Research report for external body
| Title | Research report for external body |
| Category no. | N |
| Description | Non-confidential reports, commissioned and/or funded by an external organisation, including reports for private companies, government departments and nongovernmental organisations. May also include non-commissioned reports. |
| Assessment | This is a non-publication category from the standard REF and, as such, is assessed on the basis of significance, impact and rigour. |
Confidential report for external body
| Title | Confidential report for external body |
| Category no. | O |
| Description | Confidential reports commissioned and/or funded by an external organisation, including reports for private companies, government departments and non- governmental organisations. For clarity, confidential material is not in scope of the open access requirements. |
| Assessment | This is a non-publication category from the standard REF and, as such, is assessed on the basis of significance, impact and rigour. |
Software
| Title | Software |
| Category no. | G |
| Description | Originally researched, created and published software (computer programs and their associated documentation, consisting of a set of instructions written by a programmer) or database products of commercial quality, which has been made publicly available. |
| Assessment | This is a non-publication category from the standard REF and, as such, is assessed on the basis of significance, impact and rigour. |
Website Content
| Title | Website Content |
| Category no. | H |
| Description | A collection of material which embodies research and is undertaken on a systematic basis specifically for dissemination through a website and/or as an interactive approach to allow users to engage directly with the process or products of the research. |
| Assessment | This is a non-publication category from the standard REF and, as such, is assessed on the basis of significance, impact and rigour. |
Digital or Visual Media
| Title | Digital or Visual Media |
| Category no. | Q |
| Description | Research outputs presented in digitised and/or audio-visual format. This may take the form of moving image, sonic, visual or other digital media or written text, or a combination of these, as appropriate, to enable the panel to access the research dimensions and/or the researcher’s contribution to the output and to assess its significance, originality and rigour. |
| Assessment | This is a non-publication category from the standard REF and, as such, is assessed on the basis of significance, impact and rigour. |
Research Datasets and Databases
| Title | Research Datasets and Databases |
| Category no. | S |
| Description | Data sets may come in a variety of formats, for instance in spreadsheet, but also any collection of data on which analysis can be performed. Databases are collections of data specifically organised and presented for the ease of viewing, retrieval and analysis. |
| Assessment | This is a non-publication category from the standard REF and, as such, is assessed on the basis of significance, impact and rigour. |
Translation
| Title | Translation |
| Category no. | V |
| Description | A translation of a work or body of works by another author or authors, informed by critical evaluation of the sources (such as earlier manuscripts, texts, documents and letters), and by critical analysis of the work’s original cultural context for the new readership. |
| Assessment | This is a non-publication category from the standard REF and, as such, is assessed on the basis of significance, impact and rigour. |