Skip to content

Validating how we can go beyond the article

Validating how we can go beyond the article

As part of our 18-month research and design project, funded by the Robert Wood Johnson Foundation and the Gordon & Betty Moore Foundation, we’re exploring how research publishing can be a lever for systemic change.

This spring, we interviewed 13 leaders from funders and academic institutions across Europe, North America, and Asia to understand whether the direction we’re heading in can truly drive change in how research is assessed and recognized. Here’s what we learned.

An encouraging direction of travel

Europe and the UK are leading the way. Participants described active efforts to integrate open science into assessment frameworks, including narrative CVs, guidelines on reporting non-article outputs, limitations on publication numbers to emphasize quality over quantity, and reduced reliance on journal-based metrics.

In Asia, change is only just beginning. Open access and data sharing policies are starting to take shape, but these have yet to translate into changes in assessment.
North American reforms are often local. Where changes are occurring, these are often within departments or research units, rather than institution-wide.

Opportunities from open science

PLOS-brand-image-full-waves-scaled_1500x1500

Participants highlighted many motivating factors for this shift towards open science practices in research assessment:

  • Accelerating discovery and innovation: for example, through data and code reuse.
  • Demonstrating institutional leadership in research integrity and assessment reform.
  • Strengthening compliance with funder policies: "open research is increasingly required by funders, so emphasizing this should put the institution in a strong position."
  • Improving research integrity and reproducibility: "The goal is to incentivize people to do open, transparent, reproducible, rigorous research."
  • Recognizing a broader and more diverse range of contributions, including a broader range of contributors: "for instance the instrument or facility operators and lab technicians who help the authors gather necessary materials and data, and who are often not credited in research articles."

Barriers to change

Our interviewees told us that change will be slow, identifying a number of persistent barriers:

plos-rnd-barriers-to-change-infographic-resized-cropped

Entrenched cultural norms, like the dominance of journal prestige, remain strong, even in Europe where change is further established. 

Entrenched cultural norms, like the dominance of journal prestige, remain strong, even in Europe where change is further established. 

Variance in disciplinary norms can also make it challenging to develop an assessment framework that is equitable for all researchers.

Researchers and institutions feel "trapped" in existing incentive structures. This is particularly challenging for early career researchers motivated by career progression. Meanwhile, institutions see a potential “first mover disadvantage” when it comes to competitive funding and recruitment.

Time, tools, heuristics, and expertise are further practical barriers to recognizing open science in research assessment.

Support for the knowledge stack

"What you are proposing is what the future of science should look like. PLOS are the right people to be doing this."

Participants in our stakeholder research and convenings have endorsed the knowledge stack as a promising innovation that could help disrupt the current article-centric model of research assessment. Some even likened it to earlier publishing innovations, like DOIs and ORCID, that eventually reshaped researcher behavior.

Several key findings from our initial convening were further supported:

An icon of a cog
Data and code are priority outputs: These were seen as essential for reproducibility and high reuse potential.
An icon of boxes arranged in a hierarchy
Metadata is foundational. High quality metadata was identified as a critical building block to support tracking, transparency, and credit.
An icon of arrows coming out of a box
Any new system must not add to researchers' workloads. Instead, it should streamline processes, improve discoverability, and empower more inclusive recognition of diverse contributions.

There are many questions to resolve as the knowledge stack design progresses, notably around trust signals and quality control for non-article outputs, and how to ensure compatibility with the broader scholarly infrastructure.

Collaboration is key

 

Stakeholders stressed the importance of coordination. That means involving a broad coalition of infrastructure providers, funders, and institutions to ensure that what we’re building is useful beyond PLOS and scalable across disciplines and regions.

As we continue this work, we remain committed to transparency, iteration, and collaboration on the solutions we create. We’re grateful to the participants who generously shared their time and insights. Their feedback is shaping how we design a more inclusive, transparent, and equitable future of research.