The Spanish government introduced this system in 1994 in an effort to increase the productivity of the country’s researchers. And it succeeded: By 2021, the country ranked 11th globally in scientific output, producing more than 100,000 publications annually, according to the Ministry of Science and Innovation.

But many academics say the system has had a negative impact on the quality of their work and on science in general. It unleashed “an insatiable paper publishing machine,” says Ángel Delgado Vázquez, head of the Learning and Research Support Service at Pablo de Olavide University. Researchers can work frantically to meet the goal, with some publishing an article every 2 days. Others have cut corners, publishing low-quality papers or fraudulently seeking co-authorship; some have admitted to receiving payments from researchers in Saudi Arabia or India to co-author papers they did not contribute to. Critics say the system has also encouraged researchers to prioritize publishing over teaching, and to use public funds to pay increasingly high article-processing charges imposed by some journals.

Under the new system, ANECA wants assessments to consider a broader range of research outputs, including “publications, patents, reports, studies, technical works, artistic works, exhibitions, archaeological excavations, [and the] creation of bibliographic records.” Assessors will no longer consider only the impact factor of the journals in which scientists publish, but also details such as whether the research reaches nonacademic audiences through news reports or government documents. Papers will also score more highly when coproduced with local communities or other nonacademic authors. And in an attempt to reduce the level of public funds being spent on publication costs, assessors will take into account papers published on noncommercial, open-access publishing platforms that don’t charge author fees, such as Open Research Europe.

The changes are an attempt to fix a system that has prioritized “quantity over quality” and to “recogniz[e] that are different ways of doing science,” says Pilar Paneque, ANECA’s director. She also hopes the new system will help reduce fraud in publishing. (Delgado Vázquez, for one, thinks it will “undoubtedly contribute to stamping out unethical practices.”)

The proposals are a “bold gamble,” says Núria Benítez Monforte, a research administrator at the Catalan Institute of Nanoscience and Nanotechnology. But others say even more radical reform is needed. For example, information scientist Eva Méndez, who serves on the Steering Board of the Coalition for Advancing Research Assessment, says the 6-year evaluation cycles should be abolished altogether, because they “only serve to enrich major scientific publishers, adding to the frustration of younger researchers.”

ANECA is currently reviewing some 600 comments it has received on its proposed changes, with a goal of finalizing the new assessment system by the end of the month. If all goes as planned, it would go into effect on 1 January 2024.