Before a Human Reads It: Inside the Journals Quietly Letting Algorithms Screen American Science
Photo: manuscript review computer screen scientific journal editorial office, via learn-en.org
The first reader of a manuscript submitted to a growing number of American journals is no longer an editor with a cup of coffee and a red pen. It is a model — trained on millions of papers, tuned to flag statistical inconsistencies, out-of-scope framing, and methodological red flags — that renders a judgment in seconds. For most authors, this happens silently. The rejection letter that arrives within forty-eight hours may owe its speed not to an overworked editorial board, but to software.
At KSMV, we have heard from members across the country — from large research universities to smaller teaching institutions — who are grappling with what this shift means. The questions are practical, ethical, and increasingly urgent.
The Quiet Adoption
No major US society journal has announced a sweeping AI-first editorial policy. Instead, adoption has been incremental and largely undisclosed. Several publishers now use language models to perform what industry insiders call "triage": a preliminary screen for scope fit, statistical plausibility, and image integrity before human review begins.
The rationale is volume. Top-tier American journals routinely receive more than ten thousand submissions annually. Editors and volunteer reviewers — often faculty juggling grants, teaching, and their own labs — cannot read everything. A model that reliably filters out clearly out-of-scope papers or flags irreproducible statistics could, in theory, free human expertise for the work only humans can do.
In principle, few editors disagree. In practice, the details matter enormously.
Where Efficiency Ends and Bias Begins
The central concern raised by KSMV members is not that machines will replace reviewers. It is that machines will shape which papers ever reach them.
Large language models are trained on existing corpora — and existing corpora reflect existing priorities, citation patterns, and, yes, biases. Work from under-resourced institutions, from non-native English speakers, or from fields that fall outside a model's dense training data may be systematically disadvantaged by a screen that mistakes unfamiliarity for low quality. A 2023 Stanford study found that language models evaluating abstracts showed measurable preferences for certain writing styles over others, independent of scientific merit.
There is also the problem of the model's own confidence. A statistical error flagged by a model may be a genuine flaw — or a novel method the model simply does not recognize. When a rejection is automated, the author has no recourse to explain, clarify, or defend.
What Disclosure Should Look Like
If AI triage is here to stay — and the economics suggest it is — the KSMV position is that transparency must be non-negotiable. We propose three baseline standards:
Disclosure to authors. Any submission screened by automated systems should be told so, in plain language, at the point of submission. Authors deserve to know who — or what — first evaluated their work.
Disclosure to reviewers. Reviewers receiving manuscripts should know whether an algorithm has already flagged or summarized the paper. Uncritically passed-along machine judgments risk anchoring human review in ways reviewers never consented to.
A human appeal path. No submission should be rejected by an algorithm alone. A clear route to human reconsideration — at no cost to the author — should accompany every automated decision.
Some publishers are moving in this direction. A handful of society-owned journals now include AI-use statements in their author guidelines, and a few offer opt-out provisions for sensitive submissions. But voluntary, inconsistent policies are a poor substitute for community norms.
The Societal Stakes
Scientific societies occupy a unique position here. Many of the most respected American journals are society-owned, governed by member communities with a direct stake in editorial integrity. Unlike commercial publishers answerable primarily to shareholders, societies can set standards that reflect their members' values.
KSMV's Science Policy committee is developing recommendations for member journals, and we encourage editors, authors, and reviewers to contribute to that process. The technology will not wait for consensus. But the norms that govern it are still ours to write.
Peer review has always been a human institution grafted onto a professional one — imperfect, slow, and defensible precisely because people answer for its judgments. The machines now entering the pipeline can be useful colleagues. They cannot be accountable ones. Preserving that distinction is the task before us.
