Perhaps one of the biggest shocks to students new to research is how slow and painstaking it is. More than one has decided it is no fun at all, nothing like reading cool articles or watching nature videos. And I’m not here to tell you otherwise. If anything, you should start out even more slowly and carefully. Worry about everything. Record everything. But what does that even mean?
Well, the new journal from the for-profit line from Nature (actually Springer Nature and don’t get me started), this one being Nature Ecology and Evolution has a very useful checklist. I suppose a lot of other journals have this too, but here is the one I came across. For every figure, you have to have in the caption (or methods if too long)
sample size as a number,
sample collection methods and if technical or biological replicates (what these mean with microbes can be challenging)
how many times the experiment was replicated
definitions of statistical methods and measures
for sample sizes under 5 each data point has to be plotted
clear information on tests, whether one or two sided, indication of centrality and error bars.
Then they have several other sections where you have to report exactly where in the text something was done, like sample size selection, inclusion criteria, randomization and blinding procedures, normality of data and more.
Then you have to make it clear where your data will be publicly available and how your code will be published. They encourage Data Descriptors “to maximize data reuse.” but that link did not work for me.
Plos Biology also has some lists for good standards for meta analyses and the like, here.
So pay attention to these standards when you are designing your experiments. I’ve always said when students have asked me to consider how something will look in the Methods section. This is more specific. I particularly like the requirement that if there are fewer than 5 points, to show them, not make a misleading bar graph.
I also feel like our statistical analyses are stuck in the pre-computer days and we should probably ditch them and start over, beginning with randomization tests as the standard, but that is a post for another time.