Authors: Zhenzhen Zhuang, Jiandong Chen, Hongfeng Xu, Yuwen Jiang, Jialiang Lin
Abstract: Large language models (LLMs) have significantly impacted human society,
influencing various domains. Among them, academia is not simply a domain
affected by LLMs, but it is also the pivotal force in the development of LLMs.
In academic publications, this phenomenon is represented during the
incorporation of LLMs into the peer review mechanism for reviewing manuscripts.
We proposed the concept of automated scholarly paper review (ASPR) in our
previous paper. As the incorporation grows, it now enters the coexistence phase
of ASPR and peer review, which is described in that paper. LLMs hold
transformative potential for the full-scale implementation of ASPR, but they
also pose new issues and challenges that need to be addressed. In this survey
paper, we aim to provide a holistic view of ASPR in the era of LLMs. We begin
with a survey to find out which LLMs are used to conduct ASPR. Then, we review
what ASPR-related technological bottlenecks have been solved with the
incorporation of LLM technology. After that, we move on to explore new methods,
new datasets, new source code, and new online systems that come with LLMs for
ASPR. Furthermore, we summarize the performance and issues of LLMs in ASPR, and
investigate the attitudes and reactions of publishers and academia to ASPR.
Lastly, we discuss the challenges associated with the development of LLMs for
ASPR. We hope this survey can serve as an inspirational reference for the
researchers and promote the progress of ASPR for its actual implementation.
Source: http://arxiv.org/abs/2501.10326v1
