How to judge a poster competition at a major international society

The president-elect of the Animal Behavior Society has one major duty: to run the judging of the Founders Poster Competition. This person also attends the executive committee meeting and fills in as requested. Advice is given below in the form of FAQ.

What is the ABS Founders Award? This award honors one or all founders of the ABS. Its official name is the Founders’ Memorial Poster Paper Award. Most references to this prize omit the possessive apostrophe.

Who can enter the Founders Competition? Any current member can indicate that their poster should be considered for the Founder’s Award. This prize is not only for students, though many of the entrants are students.

What exactly do I have to do? You have to arrange for judging of Founders posters that are currently presented in two consecutive evening sessions. You also have to present the award at the awards ceremony.

When should I choose judges? You cannot choose judges far in advance, because it is necessary to know both how many posters need judging and who will be attending and not presenting a poster. So, I would wait until about 4 weeks before the meeting when the lists of poster and talk presenters are available.

Whom should I choose as judges? Choose people not authors on posters because they will have to stand by their poster. Choose people who are not co-authors or advisors of any participants. The judges must be members of ABS. Include younger people as judges. Find a balance among the judges in terms of research areas and organisms. I had equal numbers of men and women, and paired a person working on vertebrates with a person working on non-vertebrates as judges. Choose only one person from any one institution, so choose no one else from your own institution.

What should I do if I find out later that one of my judges has an advisory or other role with a participant? It is essential that judging be fair, and be viewed as fair. Therefore any such person should not judge at the level where that conflict of interest is involved. But if they have already been selected, one option is to have that person help narrow the field among a set of posters she or he has no involvement with, but not participate in the ultimate decision among the top contestants. This assumes that there were sufficient posters for a two-tier judging process.

What is a good system for organizing the judging? It is best that the judges not have too many posters to judge at a time.  This year I had teams of two people judge ten posters during the day. They picked their top two or three posters. Then we assembled a short list of about ten in a brief meeting (either at 5 pm after a plenary, or right before the poster judging). All judges rated the 10 finalists from each evening. Then we gave them scores and decided on the top 4 at a lively meeting after the poster session. We had over 90 posters and so I had 4 teams of two-person judges. We had 2 sets of posters, so after the second set we both decided on that night’s winners and the overall winners.

What kind of system should I use for scoring? We decided that each person should give the finalists a 1, 2 or 3.  This is easier than ranking all of them. Furthermore, we did not specify how many should be in each category. It is hard to predict how many great posters there will be.  This flexibility worked well. Then, we gave 5 points for every top score, 3 for a middle score, and 1 for a bottom score, took the average, and sorted.  There was a natural break after the top 4 posters, making it easy to give them the awards. We also decided they were sufficiently equal to get equal prizes. To be comfortable with our system, we discussed the science behind most of the top 10 posters, what we liked, and what we didn’t like. We did this each evening. We were in surprising, but not complete, agreement, overall.

Should the judging happen entirely on the evening when the people are presenting their posters? Unfortunately, this is not feasible, since we must look at many posters and have the larger group look at all the top entries from the different teams. I think posters should stand on their own, since many people look at posters when no one is there. A poster should communicate independent of the presenter, as should the abstracts, so a lot of work can be done before the evening sessions.  Anything else is either not feasible, or runs the risk of a single top prize being chosen when we haven’t all seen all the posters.

What do I do during the meeting? Find your fellow judges and thank them. Make sure that you have the right list of participants. I did this by taking the list the organizers gave me of people in the Founders Competition, and comparing it to the list of abstracts in the program. I suppose I could have read the abstracts earlier, but I did it on the first day. I found one person not on the list, and two people on the list who were not society members and so could not participate. Then look at the posters. I looked quickly at all of them before the evening. I arranged it so I was not on one of the winnowing teams. Some people may decide to withdraw during the meeting. Other posters may not appear at their sites, so be prepared for some changes at the last minute.

Is it worth reading the abstracts? I think abstracts should tell the story in brief, beginning with the question, moving to the approach, the results, the interpretation, and the final conclusion. I went through them and gave them a quick score, 2, 3, or 4, with 4 high. Basically, if they had a hypothesis of some relevance to theory and addressed it, I gave them a 4 and figured they were in the running for the prizes. In the end, the person with the best abstract had the poster we liked the best. All 4 winners had abstracts that I gave the top mark to, but there were also top marks to 22 other posters that did not win. There were posters among the finalists whose abstracts didn’t get the top score from me. In sum, abstracts can be very informative, but cannot stand alone.

What should the criteria of excellence be? I think there is only one: the best, most creative, innovative science.  Beautiful posters are nice and they should be easy to read, or we won’t realize how good the science is.  I am not in favor of detailed grading sheets, because they are more vulnerable to vagaries of judges than an overall grade is. The winner should present and test an exciting hypothesis in animal behavior.

What about inadvertent bias? We humans need to be constantly vigilant for bias, because it seems to come naturally to us. Consider whether bias of any kind has crept into your judging. Think about whether your judges have favored one taxon too much. Think about whether there is a sex bias among your awardees. It helps to have a balance among the judges. Bring up this issue if it seems the awards are going all in one direction.

What is the prize? Fortunately, this has changed recently. We have about a thousand dollars, maximum, because we have 95 entries. This is roughly ten dollars per entrant. We can divide it up as we please. I think it may be a good idea to give a prize to everyone with an excellent poster. This year, this meant 4 prizes of $250 each.

What is the web address of the official document on this award? http://www.animalbehavior.org/ABSHandbook/animal-behavior-society-handbook/11FoundersAward/

Advertisements

About Joan E. Strassmann

Evolutionary biologist, studies social behavior in insects & microbes, interested in education, travel, birds, tropics, nature, food; biology professor at Washington University in St. Louis
This entry was posted in Scientific meetings. Bookmark the permalink.

2 Responses to How to judge a poster competition at a major international society

  1. Jason Bruck says:

    Is there anyway to get feedback on our posters so we can make them better in the future and learn from the Founders Award process. Thank you so much for putting this up.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s