I freakin’ love science, and you probably freakin’ love science too – what with all those cute animal videos and large hadron colliders and head transplants and stuff. But science has a big problem. It’s broken.
To sum it up in a hackneyed meme, the issue is this: Such journals. Many papers. Wow. And a significant proportion of those papers are – to put it bluntly – crap.
The history of scientific publishing is long and noble, designed to uphold standards in the pursuit of truth, but this 200+-year-old tradition is woefully unfit for purpose in the 21st century. There are more journals than ever before, spawning at an alarming rate like so many academic tribbles.
In order to stay alive, this ever-growing mass of peer-reviewed journals needs an ever-growing pile of willing anonymous peer-reviewers, scrutinising thousands of proffered papers to make sure each one is sufficiently original and accurate for publication. And right now it’s an act of spectacularly wilful denial by the entire scientific community to assume that this system is still working properly.
"Such journals. Many papers. Wow"
Of course, there are many good and trustworthy papers out there, produced by careful scientists and thoroughly torn to pieces and rebuilt by reviewers who have the best interests of the field at heart. (Except the Third Reviewer. They’re just a vindictive git.)
But the dirty secret is that there are plenty of papers published all the time that are carelessly done, missing crucial data, plagiarised, incompetent, genuinely mistaken or downright fraudulent. And it’s bloody hard to get a paper retracted, even when it’s demonstrably wrong.
Sometimes it’s the scientists at fault, but it also comes down to the reviewers. I’m sure that many of them are great, smart and insightful scientists, with a comprehensive grasp of the complexities of their field. But some of them are not.
I know this, because my lab head asked me to review a paper for a major journal a couple of months into my PhD. I was hungover as hell, drowning in self-doubt and I knew nothing. And neither, apparently, did anybody else who reviewed the paper so we let a fairly major error slip through the net and into the public domain. I cannot believe I’m the only one who’s done this, although I may be the only one willing to confess to it.
And this stuff matters. It really matters.
It matters because drug companies base huge R&D programmes on the basis of flawed and irreproducible papers, wasting time, money and – most importantly – lives.
It matters because publishing shoddy trials in pay-to-play journals lends scientific legitimacy to substandard or even unproven treatments.
It matters because the media loves to report on “breakthroughs”– the bigger and more exciting the better – without necessarily finding out whether the papers underpinning them stem up to scrutiny.
And this matters because policy-makers and the public use these stories to anchor their view of the world, to make decisions about healthcare and all sorts of other aspects of life.
We live in a time when a lie can go around the world before the truth has even fished its knickers out from under the bed. Well-publicised stories based on flawed research are almost impossible to reel in once they’re out and about on social media.
To pick a recent example, I’m willing to bet good money that far fewer people have read the explanation of why the maths in the recent “ZOMG! Conspiracy theories proved impossible BY SCIENCE!” paper is suspect compared to those who saw and shared the original media-friendly click-shifter.
And there are far less trivial and much more personally and politically sensitive stories. The Guardian’s own coverage of deeply suspect 2015 study on whether the epigenetic effects of Holocaust trauma can be passed onto subsequent generations consistently pops up in the top ten science stories on the site. The debunking gets a fraction of the traffic – so which version of the story do you think people will tend to believe?
In defence of my trade, I would argue that it’s far too simplistic to blame all of these problems on the media, however tempting. After all, science journalists have got to get their stories from somewhere. Overwhelmingly – for better or worse – they tend to get them from press releases issued by journals, funders and research institutions. And this set-up has gotten us all into a terrible mess.
My modest thesis is this: Scientists, research funders and journals are locked in a desperate three-way co-dependent clusterfuck where nobody is prepared to pull out first, for fear of ruining it for everyone else.
"Science funding is increasingly transactional, rewarding publications rather than originality"
I don’t think this has been deliberate on anyone’s part. It’s the natural conclusion of the system of science that we’ve come to accept as the Way We Do Things Here.
Science funding is increasingly transactional, rewarding publications rather than originality. From the perspective of research-funding organisations, it’s hard to compare the importance of research projects when deciding how to hand out the ever-more-precious pennies.
Then there’s endless talk of “impact”, as if it’s possible to peer 20 years down the line and see that a handful of blotchy microscope images or smudged gels will definitely lead to a wonder-drug. But the number of papers a researcher publishes – coupled with a handy index of journal impact factors – means that funders can at least measure something. What’s more, in these days when brand awareness is king, press offices – particularly those in smaller organisations and universities – are gasping for every breath of publicity they can get, because mentions mean money.
And, of course, scientists are human beings, living in a world powered by cash. They need money to run labs and to do experiments. You’d hardly believe it, but a tiny tube containing a few drops of viscous liquid can cost as much as a new pair of Louboutins. They also need to be paid, although for the vast majority it’s not the vast wodges of cash the conspiracists would have you believe. It takes the best part of eight years training to be a scientist, if not more, and there’s any number of more lucrative careers that would earn far bigger bucks for far less stress (and fewer opportunities for accidental acrylamide poisoning).
Finally, journals – and journalists – are not philanthropic enterprises. Big publishers make big bucks, and I can’t believe that the smaller “pay to play” journals aren’t turning a profit either. And it doesn’t hurt if one of your publication’s papers gets picked up by the big media outlets, whose lifeblood is click-happy social media sharers.
This unholy mess reached its nadir in the recent expose by biologist and science journalist John Bohannon. He revealed that how hackers are using domain registration loopholes to redirect journal websites to fake sites, then encouraging hapless scientists to continue submitting papers for publication, in some cases for money (page charges are a common feature of many academic journals, so this isn’t unusual) with no intention of sending them for all that tedious peer review.
The irony here is that there’s little incentive for this to be fixed, because the hackers make their money, the scientists get to publish papers, and the universities and funders are still happy. Everyone wins, except for science and humanity.
So what can we actually do about the situation? I have a few suggestions, and I’d welcome more in the comments.
"The Higgs boson remains stubbornly unmonetised"
For a start, there’s the capitalist approach: the only science worth funding is the science that sells. Never mind publishing papers, simply patent everything, push it into the hands of private companies and let the market decide. If research leads to something that works – be it a better touchscreen, toothpaste or mousetrap - then it will thrive. If not, then was it really any good anyway? On the plus side, this model would work great for new drugs but not so well for the Higgs boson, which - for now at least – remains stubbornly unmonetised.
Then there’s socialism. All research groups should be allocated an equal amount of money on a per-head basis, regardless of their field, and have to beg, borrow or bribe to meet any further needs. Alternatively we could imagine some kind of Liberal Democrat model, where every lab gets money for five years and then never again.
Turning to the journals, should we scrap every publication with an impact factor below 15, and set the bar higher for everyone? Ban press releases? Pre-publication sites such as ArXiv and Haldane’s Sieve are noble attempts at fixing some of the problems with shoddy peer review, but maybe we should shut down all the journals entirely and just make labs blog their results, tweeting their latest findings into the social media void?
Perhaps tedious grant-funding meetings could be replaced with some kind of televised lottery, pulling applications out of a tombola at random? (“And the bonus ball is… Mechanisms of DNA repair in the budding yeast Saccharomyces cerevisiae!”) Then finally there’s my personal favourite option: simply reading out anonymised research proposals to a bunch of nerds and waiting for the assembled group to moan “Whoah! Duuuuuude!” in their best Keanu Reeves voice when something really good hits the spot.
Whatever it is, we need to do something soon. The problems with modern science have gone beyond being an elephant in the room. They’ve become a steaming, stinking pile of pachyderm poo. We can’t ignore them any longer.
