The review process will be conducted transparently. All submissions, together with panel notes and summaries, will be published on zenodo. Individual submission scores will not be made public.

Names of the winners and highly commended submissions must not be disclosed until the winners have been notified at the award ceremony.

Names and institutional affiliations will normally be removed from submissions before review. Where a panel believes it cannot carry out its work effectively without this information, names and affiliations may be reinstated. The Panel Chair must be satisfied that doing so will not compromise the integrity of the assessment process.

Where a panel member identifies a conflict of interest, including affiliation with the submitting institution or another relevant connection, they should withdraw from discussion and decision-making relating to that submission.

Panel Composition

Panel Chairs – Lead panel discussions; ensure consistent application of assessment criteria; facilitate consensus; manage conflicts of interest.

Panel Members – Independently assess submissions; provide scores and comments; contribute to discussions and final decisions; support selection of Winners and Highly Commended entries.

Panel Advisers – Record decisions and rationale; support cross-panel calibration; document interpretation of criteria; capture lessons learned from the assessment process.

Hidden REF Principles

Assessment of both Hidden Output and Hidden Role submissions should be guided by the overarching principles of the Hidden REF:

• To include everyone and everything, leaving nothing behind.
• To recognise the diversity, richness and variety of contributions to UK research.

Assessment Process

The assessment process will take place in two stages:

Stage 1: Individual Review

• Panel members independently assess submissions and record comments and scores.

Stage 2: Panel Deliberation

• The panel meets to discuss submissions collectively.
• The panel selects:
  • One Winner, and
  • Up to three Highly Commended submissions (at the panel’s discretion).

Each of the panels will select one Winner. Panels are not required to select Highly Commended entries. These should only be awarded where the panel believes that additional submissions merit particular recognition.

Hidden Output Submissions

The Hidden REF Competition provides an opportunity to explore how a broader range of research outputs can be recognised and assessed. It also offers insights into how future REF exercises might evaluate not-traditionally submitted outputs more effectively, appropriately and equitably.

Assessment Criteria

Hidden Output submissions should be assessed using the REF output criteria:

Significance – The extent to which the work has exerted, or is likely to exert, influence within an academic field and/or in practical applications.

Originality – The extent to which the output introduces new ideas, approaches or perspectives, or is distinctive or transformative in relation to previous work.

Rigour – The extent to which the work has a clearly articulated purpose, uses an appropriate methodology, and presents convincing evidence that its aims have been achieved.

Interpreting the Criteria

Panels have discretion to interpret these criteria in ways that are appropriate to the outputs under review. Given the diversity of submissions, panels are encouraged to broaden, enrich or adapt these definitions where necessary to recognise the full value and contribution of the work.

Scoring

Panels are not expected to apply REF star ratings.

Instead, each submission should be scored on a 1 to 5 scale, where:

• 5 = strongest
• 1 = least strong

Members should consider Significance, Originality and Rigour when scoring and produce a single overall score for each submission.

Hidden Role Submissions

Hidden Role submissions recognise individuals, teams, groups or communities whose contributions to research are significant but often overlooked or insufficiently acknowledged.

All Hidden Role submissions will be assessed by a single dedicated panel.

Assessment Criteria

Submissions should be assessed against two criteria.

Visibility of the Role – To what extent was the contribution visible and recognised? Panels should identify cases where a significant contribution was not fully visible or acknowledged, either within the immediate community where the activity took place or by wider society.

Significance of the Role – How significant was the contribution to the production of research? Research often depends on contributions from individuals, teams, communities or professional groups whose work may not traditionally be recognised. Examples may include technicians, software developers, professional staff, citizen scientists, community partners or other collaborators. Assessment should focus on the significance that has been demonstrated, rather than potential future significance.

Scoring

Each submission should receive a score on a 1 to 5 scale, where:

• 5 = strongest
• 1 = least strong

Visibility and Significance should be considered together when determining an overall score.

Feed Forward and Learning

The Hidden REF Competition is intended not only to recognise hidden contributions, but also to generate insights that can inform future approaches to research assessment.

Panels are encouraged to note where existing REF criteria did not adequately capture the strengths of submissions, and where alternative interpretations or additional criteria were needed. These reflections will be valuable in informing future approaches to research assessment.

Panels are therefore asked to provide a short summary of:

• What worked well in the assessment process.
• What challenges arose.
• How decisions were reached.
• How assessment criteria were interpreted and applied.

These reflections should be provided for each submission category, rather than for individual submissions. These should be facilitated by the Panel Advisers.

Interpretation of the Guidance

Assessment within the Hidden REF is inherently subjective and deliberative. Panels should compare submissions relative to one another rather than seeking an absolute measure of quality.

Scores should inform panel discussions but should not automatically determine outcomes. 

Panel Chairs, in consultation with their panel members, may interpret and apply this guidance in ways that are appropriate to the submissions under review, while remaining consistent with the principles of the Hidden REF.

Differences of opinion are expected and should be viewed as a valuable part of the assessment process. Panels should seek consensus where possible through discussion of the evidence presented in the submission. Where consensus cannot be reached, the Chair should determine an appropriate method for reaching a final decision.

Following completion of panel discussions, Panel Chairs and Advisers will meet to consider the overall balance of competition outcomes across panels. Where necessary, they may make adjustments to the balance of Winners and Highly Commended entries to ensure a consistent approach across the competition. This may include reclassifying a proposed Winner as Highly Commended, or vice versa.

Panel members should remain mindful of potential biases relating to discipline, institution, career stage, professional background, publication format, geographical location, or socially constructed characteristics. Assessment should focus on the evidence provided within the submission and the contribution being described.