About the SA Blog Network

Doing Good Science

Doing Good Science

Building knowledge, training new scientists, sharing a world.
Doing Good Science Home

Are scientists obligated to call out the bad work of other scientists? (A thought experiment)

The views expressed are those of the author and are not necessarily those of Scientific American.

Email   PrintPrint

Here’s a thought experiment. While it was prompted by intertubes discussions of evolutionary psychology and some of its practitioners, I take it the ethical issues are not limited to that field.

Say there’s an area of scientific research that is at a relatively early stage of its development. People working in this area of research see what they are doing as strongly connected to other, better established scientific fields, whether in terms of methodological approaches to answering questions, or the existing collections of empirical evidence on which they draw, or what have you.

There is general agreement within this community about the broad type of question that might be answered by this area of research and the sorts of data that may be useful in evaluating hypotheses. But there is also a good bit of disagreement among practitioners of this emerging field about which questions will be the most interesting (or tractable) ones to pursue, about how far one may reasonably extend the conclusions from particular bits of research, and even about methodological issues (such as what one’s null hypothesis should be).

Let me pause to note that I don’t think the state of affairs I’m describing would be out of the ordinary for a newish scientific field trying to get its footing. You have a community of practitioners trying to work out a reasonable set of strategies to answer questions about a bundle of phenomena that haven’t really been tackled by other scientific fields that are chugging merrily along. Not only do you not have the answers yet to the questions you’re asking about those phenomena, but you’re also engaged in building, testing, and refining the tools you’ll be using to try to answer those questions. You may share a commitment with others in the community that there will be a useful set of scientific tools (conceptual and methodological) to help you get a handle on those phenomena, but getting there may involve a good bit of disagreement about what tools are best suited for the task. And, there’s a possibility that in the end, there might not be any such tools that give you answers to the questions you’re asking.

Imagine yourself to be a member of this newish area of scientific research.*

What kind of obligation do you have to engage with other practitioners of this newish area of scientific research whose work you feel is not good? (What kind of “not good” are we talking about here? Possibly you perceive them to be drawing unwarranted conclusions from their studies, or using shoddy methodology, or ignoring empirical evidence that seems to contradict their claims. There’s no need to assume that they are being intentionally dishonest.) Do you have an obligation to take to the scientific literature to critique the shortcomings in their work? Do you have an obligation to communicate these critiques privately (e.g., in email correspondence)? Or is it ethically permissible not to engage with what you consider the bad examples of work in your emerging scientific field, instead keeping your head down and producing your own good examples of how to make progress in your emerging scientific field?

Do you think your obligations here are different than they might be if you were working in a well-established scientific field? (In a well-established scientific field, one might argue, the standards for good work and bad work are clearer; does this mean it takes less individual work to identify and rebut the bad work?)

Now consider the situation when your emerging scientific field is one that focuses on questions that capture the imagination not just of scientists trying to get this new field up and running, but also of the general public — to the extent that science writers and journalists are watching the output of your emerging scientific field for interesting results to communicate to the public. How does the fact that the public is paying some attention to your newish area of scientific research bear on what kind of obligation you have to engage with the practitioners in your field whose work you feel is not good?

(Is it fair that a scientist’s obligations within his or her scientific field might shift depending on whether the public cares at all about the details of the knowledge being built by that scientific field? Is this the kind of thing that might drive scientists into more esoteric fields of research?)

Finally, consider the situation when your emerging field of science has captured the public imagination, and when the science writers and journalists seem to be getting most of their information about what your field is up to and what knowledge you have built from the folks in your field whose work you feel is not good. Does this place more of an obligation upon you to engage with the practitioners doing not-good work? Does it obligate you to engage with the science writers and journalists to rebut the bad work and/or explain what is required for good scientific work in your newish field? If you suspect that science writers and journalists are acting, in this case, to amplify misunderstandings or to hype tempting results that lack proper evidential support, do you have an obligation to communicate directly to the public about the misunderstandings and/or about what proper evidential support looks like?

A question I think can be asked at every stage of this thought experiment: Does the community of practitioners of your emerging scientific field have a collective responsibility to engage with the not-so-good work, even if any given individual practitioner does not? And, if the answer to this question is “yes”, how can the community of practitioners live up to that obligation if no individual practitioner is willing to step up and do it?

* For fun, you can also consider these questions from the point of view of a member of the general public: What kinds of obligations do you want the scientists in this emerging field to recognize? After all, as a member of the public, your interests might diverge in interesting ways from those of a scientist in this emerging field.

Janet D. Stemwedel About the Author: Janet D. Stemwedel is an Associate Professor of Philosophy at San José State University. Her explorations of ethics, scientific knowledge-building, and how they are intertwined are informed by her misspent scientific youth as a physical chemist. Follow on Twitter @docfreeride.

The views expressed are those of the author and are not necessarily those of Scientific American.

Rights & Permissions

Comments 11 Comments

Add Comment
  1. 1. Danliv 11:06 pm 12/11/2012

    I love this question.

    Yes, scientists have a professional responsibility to police the integrity of their field, especially in a fledgling (and therefore necessarily poorly understood) discipline.

    Failure to curb wayward research via open professional criticism, scrutiny, and peer review enables poor science. Science is not like publicity, where any amount and of any caliber is good. Any amount of poor science is detrimental to the growth and advancement of a field. There is no one better equipped or more responsible for excising inferior research from a field than peer researchers in that field. If they do not take up the responsibility no one will, and the discipline may become unabatedly riddled with pseudoscience and popular misconception, which would be egregiously harmful to a young developing field of study.

    The professional responsibility on a researcher in a fledgling field, self-servingly, is the growth and development in that field. If one does good science, but allows colleagues to disseminate misinformation unrebuked, the quality and reputation of the field as a whole is diminished. That leads to either the faltering of that field and questioning of the practical feasibility of further research, or perhaps worse a predominance of the field’s pseudoscientific popularity at the expense of further genuine science.

    If scientists want to continue doing good science, they have to call out the bad science. Professional silence lends tacit credence to the charlatans.

    That was supposed to be two cents, but it ended up a nickel.

    Link to this
  2. 2. darioringach 11:53 am 12/12/2012

    Yes, independently of the maturity of an area of research, one has a responsibility to give an honest opinion of the value and correctness of work within his/her field of expertise. Science is a community effort.

    These obligations are felt more strongly when the field in question stands to steer the course of science, the use of resources, or influence public policy. This also implies that scientists ought to speak up when they feel there is a systematic problem with poor scientific journalism.

    Link to this
  3. 3. Janet D. Stemwedel in reply to Janet D. Stemwedel 12:08 pm 12/12/2012

    So, hypothetically, if there’s an emerging area of research that has some really awful work getting published and popularized, and other scientists working in this area are, to all appearances, not saying “boo” about it, what should we conclude? It strikes me that there are conclusions we can draw about the practitioners in such a community falling down on their duties, but are there also conclusions to be drawn about the emerging area of research itself?

    Link to this
  4. 4. darioringach 12:21 pm 12/12/2012

    “…what should we conclude?”

    …I guess one should conclude they are not standing up to their duties and responsibilities as scientists and as members of society. As we know, such behavior is typical of many scientists…

    And I will admit I used to behave like that — too busy to get out of the laboratory or to be distracted form my next manuscript.

    I am not sure being an “emerging area of research” makes a huge difference on how one should behave.

    It seems you have a field of research in mind… Do you?

    Link to this
  5. 5. Janet D. Stemwedel in reply to Janet D. Stemwedel 12:29 pm 12/12/2012

    When the jaw-droppingly bad examples of evolutionary psychology are trotted out (e.g., the “work” of Satoshi Kanazawa), people sometimes raise the question of why other people doing evolutionary psychology are not more vigilant in pointing out the flawed work, both within the community and to the interested public. One of the possibilities that seems to hang in the air is that maybe they don’t call it out because their own work partakes of essentially the same methodology (and methodological shortcomings) — which would seem to be a strike against the field as a whole.

    Link to this
  6. 6. MissionHelper 5:47 pm 12/12/2012

    I don’t have answers, but those are good, meaty questions! TY!

    Link to this
  7. 7. CliffClark 8:45 pm 12/12/2012

    One of the defining characteristics of science is that it is self-correcting. In new areas of science, or even in new areas of exploration within old areas of science, it can be really difficult to distinguish truly groundbreaking ideas that make others working in the area uncomfortable from absolute crap. It may be too early to decide what is good science and bad science. One reviewer of a paper I submitted commented that nobody would be interested and that what I was working on would have no impact. In fact, the information was absolutely critical for scientists engaged in developing molecular “serotyping” methods to replace traditional Salmonella serotyping to interpret the data they got from their assays. The reviewer was ignorant, in the best sense of the word. If the science is well done,you have to let it stand the scrutiny of other scientists however you may disagree with what is published. If the science is not well done it should never pass review, though we all know that a lot of peer reviewers do not do a very good job. However, it is better to fix “the system” rather than impose censorship. Bad science should never be published but, if it is, you need to have a little faith in the self-correcting mechanism of science. As far as journalists and the public…well, if they are not capable of critically thinking about what they see and hear and read, I doubt that an authoritarian, hierarchical policing of all the data will be adequate. The people who do bad science will be found out, will lose trust and reputation, and will be selectively removed from scientific discourse.

    Link to this
  8. 8. darioringach 10:45 pm 12/12/2012

    I do not know the work of Kanazawa nor the controversy surrounding it to comment directly. But it seems to me there are so many areas of science that have much greater, immediate public impact — such as anti-vaccination campaigns, climate change and energy, “complementary” medicine, the dismal state of K-12 education … where there is a much more pressing obligation for academics to speak up and yet we see very little participation.

    Link to this
  9. 9. Postman1 8:14 pm 12/15/2012

    “Are Scientists Pbligated to Call Out the Bad Work of Other Scientists?”
    Well, yes, and to use spell check.

    Link to this
  10. 10. Postman1 8:16 pm 12/15/2012

    *I should have mentioned for the editor’s benefit, the quote was from the heading in the email.

    Link to this
  11. 11. Janet D. Stemwedel in reply to Janet D. Stemwedel 8:50 pm 12/15/2012

    Thanks for explaining that (and for the laugh)!

    Link to this

Add a Comment
You must sign in or register as a member to submit a comment.

More from Scientific American

Email this Article