Each year, this global event brings together academic publishers, institutions, associations, and researchers to celebrate the vital role of peer review in maintaining research integrity.
To mark the occasion, IOP Publishing ran a global survey to find out how researchers and reviewers feel about the intersection of AI and peer review.
In this Q&A, I will share some of the insights from the survey:
Q: Why is it important for the publishing community to consider the use of AI in peer review?
A: The last few years have seen a rapid increase in the use of consumer generative AI in almost all industries. Large language models (LLMs) are now easily accessible to anyone with an internet connection and are being used in a variety of ways. Their potential role in peer review, however, raises a number of ethical and practical concerns.
At IOP Publishing, we currently do not permit the use of generative AI to write peer review reports, either fully or in part. At the same time, we acknowledge the potential of AI to support and improve aspects of the peer review process. If we receive a report that we suspect has been written or augmented by generative AI, we let the reviewer know about our policy, and our editorial team rescinds or redacts such reports with the reviewer’s knowledge.
Given the pace of change in this field, we wanted to take a closer look at how researchers in the physical sciences currently view the use of AI in peer review, and whether there are ways in which we can use AI in the review process in a responsible way.
The outcome of our latest survey highlights polarisation within the reviewer community. 41% of respondents believed generative AI would have a positive impact on peer review, 37% saw it as negative, and 22% felt it would have a neutral or no impact.
We also saw that over a third (32%) of respondents said they had used generative AI in peer review in some form, with the majority (21% of the 32%) using it purely to improve flow and grammar. 13% said they use AI tools to digest or summarise an article under review and 2% of respondents said they used AI to write a review on their behalf. At the same time, over half (57%) would be unhappy if AI was used to write a review on their own manuscript, and 42% would be unhappy if it was used to augment a report. This contradiction underscores the need for transparency, ethical standards, and education around the opportunities and limitations of the use of AI in peer review.
Q: What is IOP Publishing’s stance on the use of AI in peer review, and how do you balance innovation with integrity?
A: We currently prohibit the use of AI in peer review because generative models cannot meet the ethical, legal, and scholarly standards required. However, we recognise the potential of AI and are actively exploring how it might be responsibly applied to support, not replace, human judgment. Balancing innovation with integrity means maintaining rigorous standards while remaining open to experimentation. One solution could be the development of AI tools embedded within peer review systems. These would assist reviewers without compromising security, confidentiality or data privacy by avoiding the risks of uploading manuscripts to third-party platforms. Crucially, such tools would be overseen by editors and designed to enhance, not replace, the reviewer’s skills and expertise.
Q: Did the survey reveal any shifts in how researchers perceive the role of peer review considering the emerging role of AI?
A: Yes, the survey revealed a growing polarisation in attitudes. Compared to our similar survey a year ago, neutrality dropped significantly (from 36% to 22%), while positive sentiment increased by 12% and negative sentiment rose slightly by 2%. This shift suggests that researchers are becoming more opinionated and polarised about AI’s role in peer review. The trust gap suggests that awareness around AI and its use is rising but researchers remain cautious about its role in evaluating scientific work, especially when it comes to their own submissions.
Q: Could you share more about IOP Publishing’s Peer Review Excellence training and certification programme?
A: Peer review takes time, care, and thoughtful consideration to provide constructive feedback. We believe that no technology currently available can replace the human skills required for high-quality peer review. To support reviewers in developing these skills, we offer free Peer Review Excellence training. The course is designed to equip reviewers with the skills and understanding needed to conduct high-quality peer reviews. It covers best practices, common pitfalls, and emerging challenges, including the ethical concerns surrounding the use of AI in peer review. It’s part of our broader commitment to improving the peer review process by supporting reviewers at all career stages and promoting diversity within the reviewer community.
Q: How does this initiative help address some of the challenges highlighted in the survey, and what kind of impact have you seen so far?
A: Our Peer Review Excellence programme directly addresses concerns raised in the survey, such as misunderstandings about AI capabilities, ethical use of technology, and quality of reviews.
For instance, several of the surveyed researchers suggested that generative AI could be used for logical reasoning and technical analysis, rather than to simply generate and edit text. For example, one researcher said: “I do not use AI for peer review. Only reading, checking reference content and reproducibility. Then analysing methods and results.” Comments such as this seem to be based on misconceptions of how generative AI works. Currently, consumer-accessible large language models are not capable of high-level logical reasoning but instead work by predicting the next most likely word in a sequence, with text outputs that make it appear that they have engaged in logical reasoning or analysis. When designing policies around the use of generative AI in peer review, we as publishers need to be clear and transparent about the true capacity of consumer-accessible AI tools. By educating reviewers, we support the reviewer community and enhance the quality of reviews.
