The End of Peer Review and The Matrix of Ideas.

(Spicer and Roulet, 2014a)

You can download and read or share a .pdf of the complete text of this essay by scrolling down to the bottom of this post and clicking on the Download tab.

The End of Peer Review and The Matrix of Ideas

What is the professional academic system of peer review? Here’s a brief synopsis:

We’ve all heard the phrase “peer review” as giving credence to research and scholarly papers, but what does it actually mean? How does it work? Peer review is one of the gold standards of science. It’s a process where scientists (“peers”) evaluate the quality of other scientists’ work. By doing this, they aim to ensure the work is rigorous, coherent, uses past research and adds to what we already knew. Most scientific journals, conferences and grant applications have some sort of peer review system. In most cases it is “double blind” peer review. This means evaluators do not know the author(s), and the author(s) do not know the identity of the evaluators. The intention behind this system is to ensure evaluation is not biased. The more prestigious the journal, conference, or grant, the more demanding will be the review process, and the more likely the rejection. This prestige is why these papers tend to be more read and more cited.

The process in details

The peer review process for journals involves at least three stages.

1. The desk evaluation stage

When a paper is submitted to a journal, it receives an initial evaluation by the chief editor, or an associate editor with relevant expertise. At this stage, either can “desk reject” the paper: that is, reject the paper without sending it to blind referees. Generally, papers are desk rejected if the paper doesn’t fit the scope of the journal or there is a fundamental flaw which makes it unfit for publication. In this case, the rejecting editors might write a letter summarising his or her concerns. Some journals, such as the British Medical Journal desk reject up to two-thirds or more of the papers.

2. The blind review

If the editorial team judges there are no fundamental flaws, they send it for review to blind referees. The number of reviewers depends on the field: in finance there might be only one reviewer, while journals in other fields of social sciences might ask up to four reviewers. Those reviewers are selected by the editor on the basis of their expert knowledge and their absence of a link with the authors. Reviewers will decide whether to reject the paper, to accept it as it is (which rarely happens) or to ask for the paper to be revised. This means the author needs to change the paper in line with the reviewers’ concerns. Usually the reviews deal with the validity and rigour of the empirical method, and the importance and originality of the findings (what is called the “contribution” to the existing literature). The editor collects those comments, weights them, takes a decision, and writes a letter summarising the reviewers’ and his or her own concerns. It can therefore happen that despite hostility on the part of the reviewers, the editor could offer the paper a subsequent round of revision. In the best journals in the social sciences, 10% to 20% of the papers are offered a “revise-and-resubmit” after the first round.

3. The revisions – if you are lucky enough

If the paper has not been rejected after this first round of review, it is sent back to the author(s) for a revision. The process is repeated as many times as necessary for the editor to reach a consensus point on whether to accept or reject the paper. In some cases this can last for several years. Ultimately, less than 10% of the submitted papers are accepted in the best journals in the social sciences. The renowned journal Nature publishes around 7% of the submitted papers. (Spicer and Roulet, 2014a)

Now, Albert Einstein despised the professional academic peer review system—and with good reason. If that system had been in place during the first decade of the 20^th century, Einstein quite probably wouldn’t have been able to publish his four revolutionary, paradigm-shifting physics papers in 1905; and today, it’s almost a certainty that he wouldn’t have. Indeed,

[i]t was only after Einstein came to the US in 1935 that he came face to face with the peer review process. He and his younger colleague, Nathan Rosen, sent a paper on gravitational waves to Physical Review, a journal which had established its reputation as the premier physics journal in the US. The paper had the potential to be highly controversial as it challenged the idea that gravitation was a wave.

John Tate, the editor of the journal, hesitated over Einstein’s paper for a month. He then send it to a reviewer for comments—his selected reviewer was probably the famously gossipy Howard Percy Robertson, one of Einstein’s colleagues at Princeton. The reviewer returned ten pages of comments which cast doubt on many of the central claims in the paper. The editor returned these comments to Einstein, asked him to consider the issues, and make any changes he saw necessary. Here is how Einstein reacted:

We (Mr. Rosen and I) had sent you our manuscript for publication and had not authorised you to show it to specialists before it is printed. I see no reason to address the – in any case erroneous – comments of your anonymous expert. On the basis of this incident I prefer to publish the paper elsewhere.

[After] he withdrew the paper from Physical Review, Einstein went on to publish it in a much more low key outlet, the Journal of the Franklin Institute. (Spicer and Roulet, 2014b)

Correspondingly, on a contemporary open-access science website, I found this cogent and compact critique of the professional academic peer review system:

The publication of articles is frequently hindered by a peer-review process that sometimes works well but far too often is seriously flawed. Journal editors are frequently not knowledgeable about a manuscript’s field of study yet have the power to reject a manuscript unilaterally. Even more problematic, biased reviewers often act as “gatekeepers” to prevent novel discoveries from being published, to the detriment of scientific progress. The flaws in the current journal publication system are having a highly detrimental effect on scientific research. A recent Nature news article (01/04/2023) reported that paradigm-shifting, “disruptive science” has experienced a massive decline of more than 90%. The article added that “the number of science and technology research papers published has skyrocketed over the past few decades, but the proportion of publications that send a field in a new direction has plummeted.” A flawed review process is one reason for this. (ScienceOpen, 2023)

Again, as Joseph Wayne Smith puts it:

the key fault with peer review I believe is the one noted by Einstein, that peer reviewers may be wrong, know less about the subject than the writer, and perhaps intentionally censor ideas that go against the present intellectual status quo. Worse still, most journal editors are lucky to be able to secure two or three reviewers, since the process of reviewing manuscripts is seldom high on professional academics’ to-do lists: their own research often becomes all-consuming in the dog-eat-dog publish-or-perish world. It is doubtful that this is adequate for quality control in any case. The result of such a system is that most academic papers are lifeless and unreadable, and seldom read as well, even if cited—using abstracts to avoid actually reading them—and most of the time, false (Ioannidis, 2005). (Smith, 2023)

Generalizing now from the formal, natural, social, or cognitive sciences to the professional academy as a whole, in my opinion, peer review is not only (i) an effective straitjacket that massively stifles creativity and guarantees in-crowd orthodoxy, but also (ii) an effective bottleneck that massively slows down and chokes the dissemination and sharing of creative unorthodox ideas. For example, and to use a famous example from early 20^th century philosophy, counterfactually projected into the third decade of the 21^st century, were Ludwig Wittgenstein, from out of the blue, to attempt to publish his brilliant, highly unorthodox, and indisputably great Tractatus Logico-Philosophicus at any respectable or highly-ranked professional academic journal or press today under the professional academic peer review system, he’d fail.[i] Full stop.

On the same open-access science website I mentioned above, the editors of a journal collection called “Airbursts and Cratering Impacts” propose to respond to the problems of peer review in the following way:

Our journal collection, “Airbursts and Cratering Impacts,” covers all aspects of impact events on the Earth by comets and asteroids. It is open-access, peer-reviewed, and multidisciplinary, and it encourages submissions on significant, cutting-edge, impact-related investigations that:

Are broadly multidisciplinary, making them difficult to review;
Run counter to a prevailing view;
Are too novel to receive a fair review; or
Have been rejected by other journals.

We support the philosophy that publishing scientific articles should be simple and easy for authors. More importantly, the significance and usefulness of new knowledge should be decided by many scientific experts rather than filtered through one editor and a few reviewers….

We are helping to counter [the flawed current peer-review process] by utilizing a multi-tiered peer-review process with both single-blind and open-review components, as follows:

First, the article is internally reviewed by our expert Editorial Board members and Guest Editors.
Second, the Board will invite single-blind reviewers to comment on the article. Previous reviews from submissions rejected by other journals are also considered and given the same weight as current reviews. Our commitment is to rarely reject submissions outright but rather to work with authors through multiple revisions until a manuscript is acceptable for publication. This assures that ground-breaking discoveries will be published rather than suppressed and are widely available to readers at no cost.
Third, the article will undergo a non-anonymous, post-publication review, in which an article’s quality and impact are judged by comments from the scientific community at large, by its number of downloads, by its Altmetric score, and by its number of citations. (ScienceOpen, 2023)

Nevertheless, I’m highly doubtful that even the multi-tiered, many-voiced, single-blind, multiple-revisions, post-publication-open-reviewed peer review process proposed by the editors of that journal collection is an adequate response to the problems of the professional academic peer review system, for two principal reasons.

First, the multi-tiered, many-voiced, single-blind, multiple-revisions, post-publication-open-reviewed character of the revised process seems to me every bit as likely to stifle creativity and guarantee orthodoxy as the existing professional academic peer review system. Whether there are only a few people telling the author(s) what to think and what to write, or many people telling the author(s) what to think and write, wouldn’t make any appreciable difference, since the author(s) would be subject to the tyranny of the few or the many in any case.

And second, the pre-publication stages of the revised process itself are so complicated that the process would inevitably roll on for months or years before anything is ever actually published. Anyone who’s ever worked inside the professional academy knows, professional academics, as a class (obviously, with some individual exceptions, and of course there are sometimes good excuses: family emergencies, illness, stress, and so-on), are notoriously slow to reply to their email, often never replying at all (aka “ghosting”), and notoriously unable to get things done in a timely manner—someone looking in at them from the outside might well even with some justification say, “gosh, they’re amazingly lazy”—and the COVID-19 pandemic has made this vocational vice even worse.

Therefore, in my opinion, we should simply shut down the professional academic peer review system altogether and forever. Instead, serious scholars inside or outside the professional academy should use large, minimally-constrained online sites like academia.edu (Academia, 2023)—perhaps in conjunction with their own personal websites—as places where anyone with a scholarly background or interests could publish their own research online and as freely downloadable, whether by means of essays, books, or journals, whether single-authored or co-authored, and also send follow-up thoughts to each other or engage in longer discussions, and then revise and re-publish, according to their own lights and at their own pace, to the best of their abilities. Then serious generalist or specialist scholars, or scholarly-minded people interested in some or another subject, could read around on such sites as much as desired or needed, and praise, use, and cite whatever they found to be good, true, or even brilliant and great work. Since there would be an absolutely level playing field as far as being published itself were concerned—everyone would self-publish—no one would be excluded from publication for bad reasons.

This proposed system of course would also entail that the administrative bean-counters inside the professional academy wouldn’t be able to use numbers of publications, or the thickness and well-paddedness of CVs, as the beans-to-count for purposes of hiring, firing, tenure, promotion, and more generally slithering up the greasy pole of professional academic career advancement. As a consequence, the administrative bean-counters would have to use quality of teaching instead as the bean-counting measure. But in my opinion, prioritizing and rewarding good and excellent teaching would actually be a very good thing from the standpoint of the core aims of higher education.

Someone might wonder whether, in the post-peer-review (for short, PPR) environment I’ve just described, there might be a tower-of-babel problem, whereby many different and mutually incommensurable or mutually misunderstanding voices talk past each other and drown each other out, so that the whole enterprise becomes chaotic or even collapses altogether. But on the contrary, I strongly believe that the contemporary professional academy already has a serious tower-of-babel problem: hyper-specialists in subject or topic X constantly talk past and drown out other hyper-specialists in subject or topic Y; and in professional academic philosophy in particular, for example, “Analytics” and “Continentals” have been talking past each other and drowning out each other for the last 70 years (Rorty, 1982; Hanna, 2021: ch. VII, 2022: section II). Moreover, I also strongly believe that in the PPR environment, liberated from the straitjackets and bottlenecks of the professional academic peer review system, the good, true, or even brilliant and great work would ultimately emerge and find a correspondingly receptive, wide audience, thereby bringing about intellectual and theoretical progress for humankind. Moreover, in the PPR environment, work that’s less than good, true, or even brillant and great, would also be noticed and celebrated less, roughly in direct proportion to the lower quality of the work. In this way, lesser work would fall by the wayside and pose no problems for anyone, except, perhaps, in the disappointment experienced by some scholars. But then, since their lower quality work wouldn’t be singled out for special abuse or criticism, they wouldn’t be publicly shamed; and no one can be rationally justifiably blamed or faulted for wholeheartedly trying and for working to the best of their abilities. Would it be possible for for some charlatans and emperors without any clothes—i.e., work that’s actually bad, false, or humdrum and mediocre, but flashy or glossy—to be highly noticed and celebrated by mistake in the PPR environment? Yes, it’s at least possible. But in my opinion, this kind of mistake would be far less likely to happen than is already the case under the existing professional academic peer review system, with all its straitjackets and bottlenecks.

But here’s one crucial caveat. By making this PPR proposal, I’m also hereby critically rejecting the mechanical, constrictive thought-shaper (Hanna and Paans, 2021) of “the marketplace of ideas,” which commodifies and neoliberalizes the whole enterprise of serious scholarship, serious research, authentically creative production, and the dissemination-&-sharing of ideas of actual or possible benefit or value for humankind (Maiese and Hanna, 2019: ch. 4). If we critically reject that bad and false thought-shaper, then we can recognize that ideas and scholars aren’t essentially in competition with one another in a gigantic zero-sum game played between inherently egoistic or self-interested agents according to decision-theoretic algorithms, as per technocratic corporate capitalism, with a few big winners and a great many losers.

Diametrically on the contrary, in the PPR environment, ideas and scholars are essentially in collaboration with one another, in a gigantic holistic, processual, and purposive intellectual-&-social system that’s collectively aimed at goodness, truth, and knowledge. So, in diametric opposition to the mechanical, constrictive “marketplace of ideas” thought-shaper, I’m proposing instead a thought-shaper I’ll call the matrix of ideas, which captures not only (i) the structured, systematic conception of a grid, but also (ii) the organic, generative conception of a womb. Above all, in the PPR environment I’m characterizing as “the matrix of ideas,” where serious scholars are collaborators collectively pursuing goodness, truth, and knowledge, and not competitors individually pursuing professional academic zero-sum bragging-rights and glory, high social status, high salaries, and coercive moralistic power over their so-called “colleagues,” there would be no commodification, mechanization, or moralization, all of which are endemic, significant problems for contemporary higher education inside the professional academy (Hanna, 2023).[ii]

NOTES

[i] As it was in actual fact, even without the professional academic peer review system, Wittgenstein had a terrible time getting the Tractatus published. Indeed, he wouldn’t even have been able to publish the English translation of the Tractatus with Routledge & Kegan Paul finally in 1922, if his Cambridge teacher, mentor, and collaborator Bertrand Russell hadn’t personally arranged it and agreed to write an Introduction (Monk, 1990: chs. 8-9).

[ii] I’m grateful to Joseph Wayne Smith for thought-provoking correspondence on and around the main topics of this essay, for encouraging me to write it, and also for drawing my attention to (ScienceOpen, 2023).

REFERENCES

(Academia, 2023). Academia.edu. Available online at URL = <https://www.academia.edu/>.

(Hanna, 2021). Hanna, R., The Fate of Analysis: Analytic Philosophy From Frege to The Ash-Heap of History. New York: Mad Duck Coalition. Affordably available in hardcover, softcover, and Epub at URL = <https://themadduckcoalition.org/product/the-fate-of-analysis/>.

(Hanna, 2022). “Six Studies in The Decline and Fall of Professional Academic Philosophy, And a Real and Relevant Alternative.” Borderless Philosophy 5: 48-130. Available online at URL = <https://www.cckp.space/single-post/bp-5-2022-robert-hanna-six-studies-in-the-decline-and-fall-of-professional-philosophy-48-130>.

(Hanna, 2023). Hanna, R. “Higher Education Without Commodification, Mechanization, or Moralization.” Available online HERE.

(Hanna and Paans, 2021). Hanna, R. and Paans, O. “Thought-Shapers.” Cosmos & History 17, 1: 1-72. Available online at URL = <http://cosmosandhistory.org/index.php/journal/article/view/923>.

(Ioannidis, 2005). Ioannidis, J.P. “Why Most Published Research Findings are False.” PLOS Medicine. 30 August. Available online at URL = <https://doi.org/10.1371/journal.pmed.0020124>.

(Maiese and Hanna, 2019). Maiese, M. and Hanna, R. The Mind-Body Politic. London: Palgrave Macmillan. Available online in preview HERE.

(Monk, 1990). Monk, R. Ludwig Wittgenstein: The Duty of Genius. London: Jonathan Cape.

(Rorty, 1982). Rorty, R. “Philosophy in America Today.” In R. Rorty, Consequences of Pragmatism. Minneapolis MN: Univ. of Minnesota Press. Pp. 211-230.

(ScienceOpen, 2023). ScienceOpen. “Airbursts and Cratering Impacts: A Peer-Reviewed Open-Access Journal Collection Covering All Aspects of Airbursts and Impacts on Earth by Comets and Asteroids.” Available online at URL = <https://www.scienceopen.com/collection/9aae92f3-66ba-4b71-a74b-51b9995c56e5>.

(Smith, 2023). Smith, J.W. “Against the Academics: Peering at the Problem of Peer Review.” Against Professional Philosophy. 18 June. Available online at URL = <https://againstprofphil.org/2023/06/18/against-the-academics-peering-at-the-problem-of-peer-review/>.

(Spicer and Roulet, 2014a). Spicer, A. and Roulet, T. “Explainer: What is Peer Review?” 18 June. Available online at URL = <https://theconversation.com/explainer-what-is-peer-review-27797>.

(Spicer and Roulet, 2014b). Spicer, A. and Roulet, T. “Hate the Peer-Review Process? Einstein Did Too.” The Conversation. 2 June. Available online at URL = <https://theconversation.com/hate-the-peer-review-process-einstein did-too-27405>.

Download