Recently, I read a few articles that have shaken my trust in the modern scientific system a bit. The first, in The Economist, is about China, where a perfect storm of bureaucracy, ambition, and money creates a system that encourages fake publications. The second, also in The Economist, addresses bias and a disturbing lack of skepticism in science. The third was a study on open-source journals published in Science, one of the gold standards for scientific publications.
A bit of background: Scientists communicate their findings with the world by publishing papers in scientific journals. Most reputable journals have stringent peer-review processes that can take months, even over a year. When a researcher submits a paper to a journal, the editor looks at the paper, sends it out to two or three other experts to get their comments and opinions, almost always sends the paper back to the author for revisions, and then the paper gets published. At any step of the process, the paper can be rejected.
Reputations of journals matter. Nature and Science are arguably the most prestigious and most visible journals–the discovery of DNA was announced in Nature in 1953. As would be expected, these journals, as well as top journals in each discipline, often have high rejection rates and a lot of great research is excluded. If a paper is rejected, a researcher will submit to another journal that maybe isn’t as prestigious. It’s also important to note that different journals have different goals and prestige isn’t everything. Journals published by the Ecological Society of America are loosely categorized by length of manuscript, for example.
Unfortunately, the modern scientific system has created opportunities for people to make money by unscrupulous means. In China, there is a lot of pressure to publish papers, as the number of publications is taken into account when scientists vie for promotions, raises, or jobs. This has resulted in rackets selling papers with fabricated data and fake journals (that charge a fee for publication, of course). China isn’t the only country where scientific fraud is rampant, but it’s a country where the situation is especially bad. The U.S.-centered article in The Economist describes flaws in the system: competitive scientists concealing data, the shortcomings of peer review, and a tendency for journals to publish “interesting” research.
An emerging frontier in science is open-access journals. Conventional journals require an expensive subscription to access papers–expensive enough that some colleges and universities will not pay for them. Open-access journals have emerged as a new source of information, but haven’t exactly been welcomed with open arms. While conventional journals pay their costs with subscription fees, open-access journals charge authors a publication fee. And while many open-access journals are well-intentioned, others exist simply to make money.
Dr. John Bohannon, the guy who started the Dance your Ph.D. Contest, generated over 300 fake papers and submitted them to open-access journals around the world in a sting operation. These papers contained many fatal flaws that a competent peer reviewer or editor should have easily caught, and would normally lead to rejection. In a nutshell, he discovered that acceptance happened far more often than rejection, and many papers were accepted with no evidence of peer review. Some open-access journals that have been criticized in the past were some of the “best” journals in this sting. For example, PLOS ONE, a popular open-access journal, provided some of the most rigorous peer review and ultimately rejecting the paper.
In many cases, publishers, editors, and bank accounts were on different continents. The author made a cool interactive map that includes all of the emails sent back and forth between the fictitious authors (himself) and the journals.
The silver lining? Scientists are skeptical of journals they’ve never heard of before, and are trained to differentiate between good and bad science. Unfortunately, once a paper is published, it can be cited almost as if it were unquestionably true. And as the second article from The Economist states, there’s a lot of trust blindly given to other scientists. And with the communication of science looking a lot like this, the public can be greatly misinformed.
My take on the whole thing: it’s scary that there’s quite a bit of bad science out there, and sad that people would exploit the system to make a quick buck. It’s worse that the bad science is in open-access journals, available for free to anyone. Not only does this erode trust between scientists, it also erodes the immense trust that the general public places in scientists. The second Economist article puts it best: “The false trails laid down by shoddy research are an unforgivable barrier to understanding.” What some people are doing goes against the very spirit of science, but perhaps they’re just not scientists.
This post is also published on my personal blog.