EMBO has long advocated for moving beyond publication-based metrics in research assessment, and towards more qualitative modes of evaluation, for a more holistic and transparent assessment of the quality of researchers and research proposals. As part of this commitment, EMBO is a signatory of both the Declaration on Research Assessment (DORA) and the Coalition for Advancing Research Assessment (CoARA). In line with their principles, EMBO seeks to ensure that its evaluation criteria support clear, transparent and fair selection decisions.
On 20 January, EMBO held a workshop to collect the input and feedback of representatives from its scientific community on possible changes to the selection criteria for the following funding schemes: Postdoctoral Fellowships, Young Investigator Programme, Global Investigators, and Installation Grants. The outcome of the workshop will be a set of recommendations for the selection committees and for the EMBO Council.
Organized by EMBO Director Fiona Watt and Senior Policy Officer Sandra Bendiscioli, the workshop included members of the EMBO selection committees, Council Chair Marta Miączyńska and other EMBO Members interested in these issues, as well as Heads and Senior Officers of the EMBO Programmes
The workshop, «Responsible Research Assessment – Revisiting the EMBO Evaluation Criteria » took place at EMBO, Heidelberg.
Evolving approaches to research assessment
While journal-based metrics are easy to use and help reviewers make faster decisions, they can reinforce the publish-or-perish culture and contribute to biases in funding decisions rather than reliably identifying the highest-quality science.
During the one-day workshop, one of the key questions was related to how to reconcile the rising volume of applications with a more qualitative mode of assessing research quality.
“The way we do research is evolving, with scientists increasingly working across disciplinary and national boundaries; furthermore, career paths are becoming more varied and less linear”, observed Fiona Watt, Director of EMBO. “At EMBO, we want to ensure that our funding schemes remain fit for purpose and continue to serve our scientific community effectively, which is why the input gathered through this workshop is so important.”
The workshop discussed case studies of real-life challenges encountered in selection rounds, exploring key questions such as:
- What do we mean by excellence?
- Are first and last authorship still appropriate eligibility criteria?
- Shall we limit the number of publications listed and hide journal names in applications?
- What additional criteria could help us capture a more holistic view of researchers, the research process, outputs, and outcomes?
- How should geographical diversity be considered in shortlisting and “grey zone” applications?
- How can reviewer workload be eased to prevent peer-reviewer fatigue?
This is the first time that EMBO has convened reviewers across all the selection committees, providing a platform for vibrant discussions. As Lea Sistonen, Chair of the Postdoctoral Fellowship Committee commented: “It was encouraging to see EMBO actively consulting members of its community on how to improve research evaluation. The polls at the end of each session helped turn initially polarised discussions into a clear sense of shared priorities and agreement.”
Highlights of the discussion
Among the many insights that emerged during the workshop, the following points were identified as particularly relevant in the review of selection criteria of the EMBO Programmes:
Management of reviewer workload
Reviewer workload emerged as a key challenge, particularly for the Postdoctoral Fellowships, which now receive more than 1,700 applications each year. With a limited reviewer pool and a broad range of research topics, participants discussed possible solutions, including testing AI-based tools, expanding reviewer pools beyond EMBO Members, and encouraging wider participation in peer review.
Rethinking the role of publications
While publications remain central to EMBO funding schemes, participants questioned the use of first- and last-author positions as formal requirements. The rise in author numbers and shared authorship mean that author position does not always reflect individual contributions and can sometimes be shaped by negotiation or conflict within research groups. Alternatives such as contributor role taxonomies (e.g. CrediT) and EMBO journal microattributions were discussed, though their scalability remains limited.
Narrative components
Participants supported introducing short, structured narratives in applications to allow applicants to describe their specific contributions. This approach, already partially applied for the Young Investigator scheme, would help reviewers assess research contributions more accurately.
Recognition of broader contributions to the scientific community
The discussion also considered recognising activities such as mentoring, peer review, conference organisation, and public engagement, particularly within the Young Investigator Network. While these contributions were seen as important for a healthy research culture, concerns were raised about added pressure on early-career researchers and challenges in assessing quality.
EMBO greatly values input from its community and will use the suggestions from this workshop as a foundation for further discussions within its selection committees and with the EMBO Council.
A report of the workshop will be made available in April on the EMBO website.
