Against the Academics: Peering at the Problem of Peer Review.

“Peer Review? No Thanks.” By T. Ehrmann (Spicer and Roulet, 2014)

About the Author:

Joseph Wayne Smith is a dissenting and/or heretical philosopher, or post-philosopher, who soon after completing his first PhD in philosophy in 1987, was unemployed, and then re-trained in other disciplines, including environmentalism, medicine, and law, eventually obtaining PhD degrees in all three fields. At the end of his quasi-career, he’s working on tearing down the crumbling edifice of professional academic philosophy before impending civilizational collapse. For more details, see: <https://scholar.google.com/citations?user=6o1_TyIAAAAJ&hl=en&oi=ao>.

You can also download and read or share a .pdf of the complete text of this essay by scrolling down to the bottom of this post and clicking on the Download tab.

Against the Academics: Peering at the Problem of Peer Review

Peer review, one of the contemporary sacred cows of professional academia, has occasionally been criticized—in peer reviewed literature—but only on a few occasions has it been regarded as irreparably flawed (Smith, 2006; Belluz & Hoffman, 2015). The conventional wisdom is that the idea of peers, informed in the subject matter of an article, act as quality control so that journals are able to publish, if not the “truth,” but something approximating it, or at least high-quality falsehoods (Ioannidis, 2005). This process is not perfect, to be sure, and fails; but the point is to improve it, the professional academics proclaim. And one way of doing this, is for experts in the various fields to be generous with their time, and devote their energies to reviewing articles. It is part of being a professional academic. Or so the story goes.

Still there are critics, most notably Albert Einstein (Spicer and Roulet, 2014). Einstein, who made revolutionary discoveries in physics early in the 20^th century, presenting the special and general theories of relativity, and laying the foundations for quantum mechanics, had only one paper blind peer reviewed. It came back with ten pages of review requirements that Einstein regarded as totally erroneous. He and his cowriter ignored this and published in another journal. Einstein’s 1905 papers on special relativity, published in Annalen der Physik, were not peer reviewed, and most important scientific papers up until the 1960s, such as Watson/Crick on the structure of DNA, were not either. For most of the history of science, and the arts, discourse was open and public (Mastroianni, 2022, a), and until the 20^th century most philosophical and artistic works, were not peer reviewed. It is interesting to speculate about the number of classic philosophical works, for example, that would have been rejected if peer review was the necessary and sufficient Big Thing in the past for publication.

This situation changed during the post World War II period, when governments began more extensively funding research and wanted an indication of quality control, to get “more bangs for their buck.” Journals begun the process of peer review, which was previously rare, and soon it became standard, across all disciplines, even the arts and humanities. Before the age of the internet, there was not much scope for wide circulation of papers, so peer review became institutionalized (Mastroianni, 2022a). What could possibly be wrong with that?

According to a searching critique of peer review by Adam Mastroianni of Columbia University, “The Rise and Fall of Peer Review,” (Mastroianni, 2022a), a paper which does not seem to have been peer reviewed—consistently enough—there is much wrong. I have greatly benefitted from the references cited in this article for motivating me to develop the critique I’m presenting in this essay. As to the failings of peer review, for a start, Mastroianni quotes an estimate that scientists collectively spend 15,000 years reviewing papers each year. This has led to a lengthy review process, because journals seldom pay reviewers and it can take some months, or years, for published papers to appear. That comes at a time where more than ever, new ideas need to quickly be made available, and the use of pre-prints has come to quietly supplement the conventional glacier-paced process of eventual publication.

Has peer review then produced the sought-after quality control? Mastroianni believes that it has not. For a start, especially for papers in the humanities, there is a low level of agreement on objective criteria for evaluation (Campanario, 1998). In fact, in areas where cultural and cognitive relativism rules the roost, such as cultural studies and postmodernist journals, the very idea of objective evaluation is rejected, as would be expected.

Even for the harder sciences, there is a replication crisis, where there is a failure to be able to replicate previous peer reviewed paper’s conclusions. For example, a Nature 2016 survey of 1,576 researchers, found that 70 percent tried and failed to replicate science experiments, with 87 percent of chemists, 77 percent of biologists and 67 percent of medical researchers (Baker, 2016). Reasons for this failure range from poor research, as a by-product of the culture of publish-or-perish (Begley & Ioannidis, 2015), to the inconvenient reality that most published research is false anyway (Ioannidis, 2005). The reasons for this plenitude of falsity range from low statistical power of studies, small samples, biases, and even fraud, which is more widespread than most academics realise (Begley & Ioannidis, 2015; Brainard, 2023).

Apart from all of the above woes, peer review often dramatically fails to produce quality control, and detect errors in papers, major errors. Use of a fictitious manuscript, sent to some leading journals, where the manuscript has major errors that should have led to immediate rejection, has been done on a number of occasions, with the similar results that the errors have not been detected; in one case, two-thirds of the major errors slipped through (Baxt et al., 1998). Cases of academic fraud also prove this point, as the fraudulent papers usually get published, and whistles are blown sometime down the road, even years later (Brainard, 2023). If peer review did what its academic supporters said it should do, then results like this would not happen.

John Bohannon, publishing under the name “Ocorrafoo Cobane,” supposedly a biologist at the Wassee Institute of Medicine, Asmara (all fictitious), submitted a fictitious paper about the anticancer properties of a chemical allegedly extracted from a lichen. Over 10-month period 304 versions of the paper were submitted, with over half the journals accepting the paper (many journals were “respectable”), even though obvious flaws were deliberately exhibited that should have been detected by anyone who could understand a basic data plot (Bohannon, 2013). Peer review spectacularly failed.

Then there is the problem of “paper mills,” i.e., organizations who produce bogus manuscripts with fabricated and/or plagiarized data, backed by dishonest reviewers, so that authors, living under the quite tyranny of the “publish or perish” regime, can inflate their CVs. Neuropsychologist Bernhard Sabel, found using his fake-paper detection methods, that in a sample of 5,000 papers, approximately 34 percent of neuroscience papers, and 24 percent of papers in medicine, in 2020, were fake (Brainard, 2023). This problem has been so troubling that some publishers have shutdown journals that had become corrupted by these paper mills. Twenty publishers are working to develop an “Integrity Hub,” producing tools to detect fake papers (Brainard, 2023, 560). The problem here is that there are many publishers who get a fee from authors for free to read access to papers, and thus have a financial incentive not to rock the publication boat. Authors, especially young ones facing the “publish or perish” imperative, have an incentive to engage in such sharp practices. The fault here does not lie specifically with peer review, but what this does show is that the peer review process can be easily corrupted and swept along with the sharp practices. Any alleged quality control of peer review can dramatically fail.

Can these failings of peer review be overcome? Most critics think that they can, but Mastroianni argues that all of the proposals have been tried, and have not improved the system—for example, paying peer reviewers, which is highly problematic when journals face rising expenses merely existing (Mastroianni, 2022b) However, the key fault with peer review I believe is the one noted by Einstein, that peer reviewers may be wrong, know less about the subject than the writer, and perhaps intentionally censor ideas that go against the present intellectual status quo. Worse still, most journal editors are lucky to be able to secure two or three reviewers, since the process of reviewing manuscripts is seldom high on professional academics’ to-do lists: their own research often becomes all-consuming in the dog-eat-dog publish-or-perish world. It is doubtful that this is adequate for quality control in any case. The result of such a system is that most academic papers are lifeless and unreadable, and seldom read as well, even if cited—using abstracts to avoid actually reading them—and most of the time, false (Ioannidis, 2005).

What should be done, and what is occurring anyway, given the built-in limitations of peer review, is to return more to the open public system of the past, this time with the internet solving the publication and distribution issues, and thus to reject an academic monoculture (Mastroianni, 2022b). Authors could seek out reviews that are not blind, and add them to the paper as appendices, and let the community of readers decide what is right and wrong. The more comments and debate the better, and readers would not be deceived about the article, having ready access to open criticisms from the conversation of humanity. And, it is pretty much what is evolving now, as conventional journals are too slow to keep up with the fast pace of knowledge advancement, not just in science, but in public policy debates. That which survives wide public scrutiny, can ultimately be published in journals, which will come to serve a more archival role, rather than leading knowledge advancement.

Robert Hanna has argued that the narrow specialization of contemporary professional academic philosophy, arising from the discipline’s uncritical and unrelenting adoption of professionalization, especially including careerism, has poisoned the discipline (Hanna, 2022). Much the same occurs across the other professional academic fields (Smith, forthcoming). Peer review, a gate-keeper and regulatory mode of regimentation of thought, is but a symptom of a deeper and wider problem, the problem of academic professionalism (Hanna, 2022). It deserves a sustained critique. For a good preliminary critical study in this direction, published 23 years ago, see (Schmidt, 2000). But the ever-increasingly rapid downward spiral of the professional academy over the last two decades, together with the recent Hun-like appearance of ChatGPT and other Large Language Models (LLMs) or chatbots at the gates of the professional academic world, make a new sustained critique of academic professionalism absolutely imperative.

REFERENCES

(Baker, 2016). Baker, M. “1,500 Scientists Lift Lid on Reproducibility.” Nature 533 (7604): 452-454.

(Baxt et al., 1998). Baxt, W. G. et al. “Who Reviews the Reviewers: Feasibility of Using a Fictitious Manuscript to Evaluate Peer Review Performance.” Annals of Emergency Medicine. 32 (3): 310-317.

(Begley & Ioannidis, 2015). Begley, C. G. & Ioannidis, J.P. “Reproducibility in Science: Improving the Standard for Basic and Preclinical Research.” Circulation Research 116 (1): 116-126.

(Belluz and Hoffman, 2015). Belluz, J. & Hoffman, S. “Let’s Stop Pretending Peer Review Works.” Vox. 7 December. Available online at URL = <http://vox.com/2015/12/7/9865086/peer-review-science-problems>.

(Bohannon, 2013). Bohannon, J. “Who’s Afraid of Peer Review?” Science 342 (6154): 60-65.

(Brainard, 2023). Brainard, J. “Fake Scientific Papers are Alarmingly Common.” Science 380 (6645): 568-569.

(Campanario, 1998). Campanario, J.M. “Peer Review for Journals as it Stands Today—Part 1,” Science Communication 19 (3). Available online at URL = <https://doi.org/10.1177/1075547098019003002>.

(Hanna, 2022). Hanna, R. “Six Studies in the Decline and Fall of Professional Academic Philosophy and a Real and Relevant Alternative.” Borderless Philosophy 5: 48-130. Available online at URL = <https://www.cckp.space/single-post/bp-5-2022-robert-hanna-six-studies-in-the-decline-and-fall-of-professional-philosophy-48-130>.

(Ioannidis, 2005). Ioannidis, J.P. “Why Most Published Research Findings are False.” PLOS Medicine. 30 August. Available online at URL = <https://doi.org/10.1371/journal.pmed.0020124>.

(Mastroianni, 2022a). Mastroianni, A. “The Rise and Fall of Peer Review.” Experimental History. 14 December. Available online at URL = <https://www.experimental-history.com/p/the-rise-and-fall-of-peer-review>.

(Mastroianni, 2022b). Mastroianni, A. “The Dance of the Naked Emperors.” Experimental History. 28 December. Available online at URL = <https://www.experimental-history.com/p/the-dance-of-the-naked-emperors>.

(Schmidt, 2000). Schmidt, J. Disciplined Minds: A Critical Look at Salaried Professionals and the Soul-Battering System That Shapes Their Lives. New York: Rowman & Littlefield.

(Smith, 2006). Smith, R. “Peer Review: A Flawed Process at the Heart of Science and Journals.” Journal of the Royal Society of Medicine. 99 (4): 178-182.

(Smith, forthcoming). Smith, J.W. Cognitive Psychology and the Epistemological Crisis, Lewiston MN: Edwin Mellen Press.

(Spicer and Roulet, 2014). Spicer, A. and Roulet, T. “Hate the Peer-Review Process? Einstein Did Too.” The Conversation. 2 June. Available online at URL = <https://theconversation.com/hate-the-peer-review-process-einstein did-too-27405>.

Download