Do you deserve to be told why your preproposal was not invited?

IMG_0763

Don’t give up science for sand sculptures just yet.

By now everyone has heard whether or not their preproposal at NSF in DEB or IOS has been invited for a full proposal. About three quarters of us are disappointed, perhaps fewer if people wrote more than one, so had better odds. What should you do if you were not invited to go ahead? What I have to say is not specific to a narrow area or the USA but is generally relevant, using this system as an example.

First of all, remind yourself of what a great scientist, teacher, and mentor you are. Do not fall into the trap of low expectations for yourself. This is really hard to do when you have had a punch in the gut from NSF, particularly as you face the real consequences for funding your lab, getting tenure, and retaining respect at your institution. Don’t let them take away your self esteem. Don’t let them make you feel terrible. Figure out somewhere to get satisfaction anyway. This often comes best from remembering to give. Find something to do for someone, in science, teaching, or otherwise.

But ultimately you do have to look at that proposal and the comments. Ideally these comments will help you write a more effective preproposal next January, or for another program. They might like some of your plans but not others. They might have discovered a methodological flaw. They might think your sample size is too small, or your approach inadequate. Read these comments really carefully. Try to see your proposal through someone else’s eyes. Get someone to read it outside your group. Pretend it isn’t your proposal but someone else’s whom you are mentoring.  Set defensiveness aside.

But, you say, you did not get a real review. There were no substantive comments. The referees did not like the proposal, clearly, but only said things like it wasn’t cohesive, there was too much information on the system, or too little, too many hypotheses, or too few, the aims were too tightly connected, or unconnected. Maybe you didn’t even get the full three reivews that NSF says preproposals should get. You got scientific word salad. The reviewers did not want to bother to tell you exactly what was wrong with your proposal. They didn’t like it. They liked others better. They didn’t bother to do the hard job of careful comments, but it is clear your proposal did not excite them.

Do they owe you a real review? Are they required to actually grapple with the ideas in your proposal and point out where they are strong and where weak? Do they have to find a flaw in your thinking to deny you? Are they supposed in any way to help you do better science?

The sad truth is that they are not. There is nothing in the whole process that says they have to actually discuss your ideas. They may be very distant from your field. All they are required to do is pick a quarter of the proposals to go forward and write some word salad for the rest. You do not even get to have three reviews necessarily. But remember, part of the reason they went to preproposals is the difficulty in getting careful reviews.

At NIH the system is different. They do not have preproposals but they triage as many as half the proposals, sending them away with no review at all. Michael Eisen just posted on Twitter that the work that got his HHMI renewed was not discussed at NIH, , @mbeisen. So, no, you do not get your work discussed there necessarily either.

Is this fair? Does the system work? I would have to say that no, it is not fair, but the system works. There is no time to be fair. The system has checks at NSF in the form of Committees of Visitors, who oversee the process after the fact. Does that make it fair? What does fair even mean with such low funding rates? This question and the general one of fairness should be addressed most explicitly with respect to systematic bias towards those underrepresented in science. This has been addressed and discussed in places like the DEB blog.

What should you do? First, remember to be the careful reviewer when it is your turn to do that. On your own proposal, call your program officer. See if she has any more substantive advice. Volunteer to serve on a panel if you have not done this, so you can see how the process works. Get colleagues to read your proposal, colleagues willing to do the hard work of being very frank and critical. Get examples of successful preproposals from friends.

One thing the preproposal process has introduced is a stage with no expert ad hoc reviews, just a panel that has to get through a lot of proposals. So the sad truth is they did not like yours and they are not the ones to tell you why.

Advertisements

About Joan E. Strassmann

Evolutionary biologist, studies social behavior in insects & microbes, interested in education, travel, birds, tropics, nature, food; biology professor at Washington University in St. Louis
This entry was posted in Grant proposals and tagged , , , . Bookmark the permalink.

7 Responses to Do you deserve to be told why your preproposal was not invited?

  1. Michelle Elekonich says:

    As a PO I can tell you we are required to have 3 reviews before we can make any decision. If someone cannot access 3, they should contact their PO

    • Does the panel summary count as a review? I did only get 2 that I can tell.

      • OK, so it turns out there was a third review that I did not get access to. The Program Officers cannot see what we get, only what they have. Three reviews are required, so if you only got two, then definitely contact your Program Officer. Mine sent me the third review by email.

  2. C says:

    I do not even work in biology, but as always, I find your posts on the academic system thought-provoking. Many of the issues with proposals apply as well for publishing. It is true that you need to set aside defensiveness. I would add to focus on the fact that of course, your work is good. You “just” need to explain better to the reviewer why it is good. You are the world-wide expert on your topic. You gotta help the rest of the world understand, too.
    Your blog is one of my favorite reads!

  3. Pingback: Friday links: Bob Paine passes away, and more | Dynamic Ecology

  4. Perfect post, Joan, and like Meg Duffy I think your second paragraph is especially important. All of us get rejected, repeatedly, and it doesn’t make us bad scientists (although it may give us ideas for how to communicate better next time). Peter Higgs’ paper predicting what we now call the Higgs Boson was rejected. The book manuscript for Gone With The Wind was rejected many, many times. Rejected? You’re in good company. (Not that any of this means we actually _like_ being rejected…)

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s