The following is partly a case study of how research can
take a wrong turn in modern molecular biology/biochemistry and partly a
reflection on the sociology of the science.
In the current scientific culture it is considered a big
deal to publish in particular “high profile venues”, such as the
Science and the Nature magazines. We can attest based on personal experience as
well as reports from our peers that publication in these venues often entails
enormous difficulty to the researchers involved on account of the peer review
practices at these venues. Thus, pushing a paper through to these venues can
earn one a virtual badge of a survivor or even hero of a great battle. This
badge confers tangible benefits in the modern scientific system: 1) the tenure
decisions of scientists at many institutions is favorably influenced by such
magazine-publications. 2) The scientific productivity reviews of researchers at
many institutions often receives a great boost from such publications. 3) A
scientist stands to get press, awards, and higher visibility (as measured by
citations) – more generally “fame” from such publications. 4) Last and perhaps
most importantly in academia, such publications could be a big factor in earning
grant money for continuing research.
Unfortunately, this system of incentives makes the
magazine-publication an end in itself, ahead of the actual science. In
the below case we illustrate how this, together with the system at the magazines, can
engender bad science.
Recently a paper was published in the Science magazine
entitled: “A nuclease that mediates cell death induced by DNA damage andpoly(ADP-ribose) polymerase-1”. Given our interest in novel biochemistries and
our long-standing investigations of self-inflicted nucleic acid damage in
programmed cell death our interest was piqued by this paper. A closer look soon
revealed that the paper had problems. Briefly:
MIF, a protein with previously reported tautomerase
activity, is a member of the tautomerase superfamily which does not feature
nucleases. The authors suggest that MIF is a DNase by claiming a structural
relationship to nucleases of the Restriction endonuclease (REase) fold, which
frequently but not always contain a PD-(D/E)XK motif. They claim that MIF
contains three copies of this motif implying that it contains three copies of
the REase fold. However, none of this is supported by structure or sequence
evidence: 1) A DALI search with the structure of MIF does not recover any REase
fold structures with Z-scores suggestive of genuine relationships (Z>3); as
expected it recovers several tautomerase superfamily structures. 2) The REase
fold is topologically unrelated to the tautomerase fold, which emerged from an
internal duplication of a simple two-strand-helix unit. 3) REase fold catalysis
requires residues (not conserved in MIF) beyond the metal-coordinating acidic
residues of the PD-(D/E)XK. Moreover, the aspartates and glutamates identified
by the authors are often on opposite ends of different structural elements and
not proximal to coordinate a metal ion. 4) These motifs, unlike the catalytic
prolines of the tautomerases, are not well-conserved even among animal
orthologs.
Further, the authors claim glutamate 22 to be the
catalytic residue equivalent to that of the so-called
Exonuclease-Endonuclease-Phosphatase (synaptojanin-like) domain (Pfam:
PF03372). This fold is unrelated to both REases (so-called PD-(D/E)XK) and
tautomerases. Their claim that MIF contains a “CxxCxxHx(n)C Zinc finger” (the Rad18
Zn-finger present in FAN1/KIAA1018) is also untenable as the side chains of
these residues are nowhere proximal to coordinate a Zn-ion in MIF. Hence, this
proposal of DNase function in MIF is based on a flawed structural hypothesis
and should be viewed with utmost caution. We have explained the above in great detail in this
preprint available from bioArxiv.
Given the serious problems with this published paper we
decided to resort to the standard process of post-facto scientific engagement.
Within 10 days of reading the said paper we submitted a technical comment along
the lines of the above-linked preprint to the Science Magazine (10/26/2016)
detailing why the paper is problematic. We believe that this was important
because the extraordinary claims made in the article flew in the face of the
foundations of biology, i.e. the evolutionary theory, and we felt this should
be made apparent to the research community. After more than a month (12/6/2016)
we heard back from the editor at the Science Magazine who had handled the
original article. Despite all that time spent we received no peer reviews of our
technical note. Rather the editor decided that even though “your discussion of
our recent paper is interesting” it was not worth publishing. However, the
editor suggested that we submit a short summary of our note as an “eLetter”
(which allows only 500 words; similar to the summary we provide above). These
are non-peer reviewed comments that are posted below the article on the Science
Magazine website at the discretion of the editor. They are neither visible in
any obvious way with the article nor are they available on Pubmed, which is the standard resource used by researchers to find literature of relevance. We followed
the suggestion of the editor by posting an “eLetter” (12/9/2016); eventually,
the editors decided to post our comment on the magazine website (12/20/2016).
Thus, almost 2 months after the original submission some form of dissenting
commentary appeared on the magazine.
So what are the lessons to learn from this story
regarding bad science? We see that there are three parties, each of which has
to be blamed for different even if somewhat overlapping reasons:
1) The authors. They are to blame because they
have utterly disregarded the foundations of biology in planning their
experiments. In any other science, like physics, a researcher is unlikely to
have any chance of being accepted as a serious scientific player in the
community if s/he were unaware of the basic foundations of the science, like
say classical mechanics, leave alone publish in the Science Magazine. However,
unfortunately, in biology several researchers can spend their publishing career
without having any more than a sketchy grasp of the foundations of biology, i.e.
the evolutionary theory, and how it is applied to study the functions of
biomolecules. This is exactly what we see with this paper. The authors blithely
disregard very basic principles of protein structure and sequence evolution to
form their starting conjecture which they then go to support with wet lab
experiments. Now, if the wet lab experiments supported their results then we
have reason to be suspicious of the way they were done – contamination, poor
controls, or even worse, some kind of unscrupulous practice. There are indeed
suspicious features regarding the experimental result of nuclease activity as
pointed out in our preprint.
If this were not enough, the authors responded to our
eLetter on 12/24/2016 as can be seen on the Science website. This response
betrays not just a lack of understanding of the evolutionary theory but even
more fundamental issues. Though the authors aligned a monomer to claim the
presence of the so-called “PD-(D/E)XK” motif in MIF in their original paper,
now they claim that the REase fold is seen only in the trimer of the
Tautomerase fold! This is not merely an about-face regarding their original
inference but indicates an even deeper lack of an idea of what comprises a
protein fold. In conclusion, based on this all we can say is that they are
either visually challenged or lack discernment regarding elementary geometric
issues such as symmetry.
2) The reviewers. The Science Magazine
helpfully provides the following details regarding the review of this paper: It
was handled by a single editor who had it reviewed by three full reviewers and
one advisor by the single blind method. The whole process took from 11/02/2015
to 8/22/2016 with two rounds of review prior to acceptance. This is an extraordinary overkill both in terms of time spent and number of reviewers for an article that should have been rejected outright in the first round of review itself or by the advisor if such were consulted by the editor before formal review. Why did
this not happen? Given the point we make above, we fear that sadly the reviewers
too, like the authors, lacked the basic qualification i.e. sound knowledge of
the implications of the evolutionary theory as applied to biomolecules.
Instead, as is typical of such magazine venues it appears that the reviewers
sent the authors around for almost 11 months on a wild-goose chase of doing
more experiments which are utterly worthless given that the starting premise
itself is flawed. What this also shows is that at magazine venues reviewers are
wont to giving trouble to authors for irrelevant things while not really
focusing on the key scientific issues in the paper. It is a certain mentality
which is sadly not uncommon among wet lab scientist where technical issues take
center-stage before asking whether the science could meaningful (and useful) or
not.
3) The Editors. Given the vastness of the
field and literature and the technical expertise needed for things like
sequence/structure analysis we would not blame the editor overly for missing
the bad science in the original submission. Nevertheless, at venues like the
magazines and high-profile journals the editors are typically former scientists
themselves. Hence, we would expect that in the least they would have some quick
intuition for good versus bad science. From personal experience we can say that
editors at such venues often fail to see merit in genuinely good and
interesting science submitted to them, while taking bad pieces like this one.
Thus, we may say that the intuition for discriminating between submissions
might be weak among the editors. However, even more damaging is their failure
to get proper peer reviews as doorkeepers of these magazines which, as noted
above, play an important role in the career of researchers. Finally, we believe
that the editor’s unwillingness to properly publish a technical comment that
reveals why the article amounts to bad science amounts even further damage to
science. Although the editors did finally post an eLetter from us, as noted
above, this is not visible via the regular channels of literature search.
Hence, no corrective is available to go hand in hand with the original article.
Thus, this move on part of the editor strikes against the much valued
“self-correcting process” that exists in science.
In conclusion, while we are not involved in any kind of
design of science policy, we still have a few recommendations to make.
- First, relates to biology education. Modern biology education necessarily needs to go hand-in-hand with proper teaching of the evolutionary theory as it applies to biomolecules, along with the accompanying biochemistry that is needed to properly understand it. Technical skills with handling laboratory equipment and experimentation, however important, cannot be privileged over such education in the above-stated fundamentals.
- Second, the scientific status-measuring apparatus needs to go slow on emphasizing publications in magazine-like venues as a “badge of honor”. Publications at such venues are short, thereby giving little space for detailed scientific description that helps develop a foundational argument properly and spot flawed ideas. Their peer-review system is aimed more at “causing sufficient trouble” rather than providing honest peer comments on the science.
- Third, the members of the editorial system should come out of the echo-chambers fostered by certain “big-name researchers” or artificially constructed “hot science” and pay independent attention to the literature from diverse journals in their respective fields.
