By now you’ve heard your fate on the preproposal. Some very generous bloggers are telling the story from the panelist perspective. I think the biggest question is whether they are treated like a four page version of a full proposal, or like a different creature, with a focus on ideas, not methods.
Jabberwocky Ecology had some encouraging news here on the big idea focus and thought it went well. Their first entry was here and in it they say that with shorter proposals more time was spent reading and thinking about them. Here’s another entry here from Jack Williams on Contemplative Mammoth. Jack gives a lot of interesting details on what they considered. I found this statement to be discouraging: “However, in general, I was impressed by how rarely the reputation of the investigators directly affected panel decisions and how consistently the review process focused on the merits of the pre-proposal in question.” I think the reputation of the researchers should influence the review process, in addition to the actual proposal at hand. After all, there are tons of studies that say this is the best predictor of excellent, published research. Of course, the two go hand in hand, but I don’t see why we can’t count previous productivity, making allowance for newcomers. Prof-Like Substance, here, also gave some first-hand insights into the process that are useful. He says big ideas are no more important than before, along with a lot of very helpful numbers.
I wonder if these preproposals were easier to evaluate than the ones I saw when I sat on a FIBR, Frontiers in Biological Research, I think, panel with preproposals years before. They were from a huge area, making us worry if us panel members were close enough to the work, given there were no ad hocs. That does not seem to have been a problem here.
I suppose you are wondering what happened to me personally, so I’ll tell you. One of our proposals got the go-ahead. The other did not. The one that was nixed was for methods in large part. It also complained that I mentioned blogs as part of my broader impacts. Go figure.
I guess overall the conclusions are still unclear. I’m hoping Jabberwocky is right and the new system will be better than the old. At least it is far more open than the NIH system.