Why you shouldn’t say “data not shown” or “personal communication”

n1026872938_30199725_7354598What makes something science is not so much the subject matter as the process. Scientific information is obtained by clear methods that others should be able to repeat. It is above all based on evidence. There are lots of different kinds of evidence and different ways of analyzing and interpreting it. Show what your evidence is. Tell how you analyzed your raw data. Explain how you interpret these data in light of theory. This is the scientific approach. It is not magic. It is not vague. It is not private.

It is susceptible to disagreement if others disagree with your methods, your analyses, or your interpretations. Part of that disagreement should not be because you had access to something secret, or your critic had access to something secret.

This is why you need to resist the temptation to refer to data you are not sharing at this time, or to communications that are not public. Science does not work by deferring to authority. It advances with data, analysis, and interpretation. These three things reveal theories that are supported. These three things are the way in which we reject theories, no matter how eminent the person advancing them is.  This is a wonderful thing about science. It is the source of all its power.

But life is complicated and you may have a piece of a puzzle that impacts a publishable study that itself is not quite ready for publication, or that belongs to a different set of co-authors. You may want to say personal communication to acknowledge an idea that was not yours. But you should not do either of these things. Restrict your story to the facts that are revealed. Save the rest for later.

I really like it that the US Proceedings of the National Academy of Sciences, PNAS, explicitly disallows these practices: “Data not shown and personal communications cannot be used to support claims in the work,” from PNAS editorial policies.

Advertisements

About Joan E. Strassmann

Evolutionary biologist, studies social behavior in insects & microbes, interested in education, travel, birds, tropics, nature, food; biology professor at Washington University in St. Louis
This entry was posted in Data and analysis, Ethics, Publishing your work, Writing and tagged , , , , , , , , , , . Bookmark the permalink.

6 Responses to Why you shouldn’t say “data not shown” or “personal communication”

  1. I really like that policy too. I also imagine that “personal observation” largely falls under this topic as well.

  2. Yes, personal observation is also one of those things you shouldn’t do. It is frustrating and hard, because we often see things that aren’t fully backed up. Just save them for later. Good science can wait.

  3. Liz Haswell says:

    This is now disallowed in many journals. It’s a hard line to walk, though, because reviewers will often ask for tangentially related experiments that don’t have a place in the current manuscript but that they still want referred to–but the journal doesn’t want anything that’s not in a figure.

  4. Owen says:

    Is unpublished data really secret? Obviously, the only reason you can cite unpublished data is because it exists and presumably you would share it if somebody asked. If others are nice enough to share their data and you cite personal communication, presumably they would share their data with others as well.

    It seems to me that personal communications and citations of unpublished observations are a means by which scientists can tip off their colleagues to productive avenues of research. It is like saying, “hey, I have private access to this proprietary information and could use it for my benefit only, but I am sharing it with you. Maybe you can make better use of this information than I can…”

    In other words, it is useful where a particular scientist has some preliminary observations but knows that he or she is not going to follow through. Sometimes it is the preliminary stuff that is the most difficult… so really this can be pretty generous.

  5. AgroEcoProf says:

    I have to say that I disagree. I think citing “pers. obs.” or the like is a method of transparency–it allows the reader to gauge for themselves what weight to give an item. It is possible that disallowing “pers. obs.” means people will no longer mention the data or observation at hand, but it seems more likely to me that it will be included as either an uncited statement or some other insinuation. I suppose it depends on the field, journal, and researcher, as well as the centrality of the point being made–if it is not a featured point or controversial assertion, though, I suspect an entirely uncited version of the same sentence could get through.

    Not to mention that it seems to me to mitigate against including items harking to the naturalist past in my field of ecology. I would rather someone include a weakly-supported (i.e., pers. obs.) idea that could be followed up on by me or other researchers, or given the low weight it deserves, or elicit another researcher to say “I had the same observation, but it wasn’t worth writing a paper about at the time! How ’bout that!”. And I don’t consider it a higher “call to authority” than the rest of the paper. After all, when you “Show what your evidence is; Tell how you analyzed your raw data. Explain how you interpret these data in light of theory”, we are taking “your” word for it that your evidence is as you present it, that your raw data is faithfully inscribed, and that the theories you’re referencing and citations you use indeed say what you say they say. Obviously, knowledgeable readers will know if something directly in their area is “off”, but how many of us are knowledgeable about every method, citation and theory in the papers we read? We can in theory (no pun intended) check for ourselves, but in practice we do not if it’s not central to our own curiosity or work. There’s simply not enough time for all the “checking”; if we didn’t rely on authority and social cues to reliability to a non-zero extent, we could do nothing but spend our times verifying.

    We already are relying on trust and authority in believing the journals we read and the authors in them. We all have certain authors we are more inclined to consider “careful scientists” and others we are inclined to view with caution; we all view certain journals as being better than others (even if we are judging more rigorously than by “Impact Factor”). Giving us more information, and more ideas, even if “half formed”, is not, to my mind, a bad idea. Making it clear that the idea is not yet “fully scientific” (analyzed and presented in the current pub) is, to me, the correct path to the transparency called for in science, while allowing the more timely disclosure of potentially interesting information.

  6. jeff smith says:

    The only time I’ve been tempted to use “personal observation” or “data not shown” is to refer to things I noticed while doing experiments that weren’t part of the original design but may have had an effect. Something like “despite being incubated in full overheard light, Dicty slugs were not completely stationary”. I didn’t collect data on slug movement; I just noticed that they did sometimes move. Those observations aren’t a major part of the argument I’m making in the paper, but they might affect how you’d interpret of some of the data. One approach would be to say that they *may have* moved and leave out the part where you know they did, but that doesn’t help transparency. In this case, writing “personal observation” or “data not shown” is basically just explicitly labelling a bit of data as anecdotal, much like what AgroEcoProf says above.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s