How scientific and fair can faculty hiring be?

OLYMPUS DIGITAL CAMERA

If you have a committee of 6 choosing the 4 or so biologists to interview for a single position, what would a really accurate way to do it be? I think you would look hard at the error factor. If smart, unbiased professors are doing the choosing, shouldn’t they arrive at broadly similar conclusions? What are the odds that even the first sorting of the long list into a shorter list for everyone to look at will be scientific?

There are several kinds of errors to consider. One I really worry about is bias. Are we reading women and minorities differently? Being aware of bias helps, I think.

Another error comes from reading files too quickly. You might miss something important because you have so many files to read and there is a deadline. It is important to divide up files so people don’t read too many too quickly.

A third kind of error comes from weighting anything besides scientific excellence and promise of future success on the first pass. We need to cut down the pool by considering this crucial factor alone.

The fourth kind of error comes in especially in really broad searches. It can be difficult to compare very different kinds of people. I was dumbfounded at how little our top neuroscience candidates publish, but that is apparently standard for their field. If they were in our search, we would not choose them, possibly missing the best person. Even in a more narrow search like ours for an ecologist, different fields are hard to compare. Theoretical vs. empirical foci, for example, result in different levels of publishing. Women are generally known to publish fewer but  more substantive papers.

We could search much more narrowly of course, for an ecologist that studies the mites on buffalo, for example. We would have a better chance of picking the best person that does exactly this, but we would miss brilliance in nearby fields.

How do we overcome all these potential errors in choosing? We don’t. We can’t be fair. In fact, one might think it is not our job to be fair since it is so impossible. Our job is to hire an excellent scientist, colleague, and teacher. There are likely to be others even  better in the pool, but not discoverable by our imperfect techniques.

IMG_8998

What having 8 people reading 60 folders and choosing the top 8 might look like.

Wow! That sure is sobering. How can I be so sure? Well, for starters, say we took our 200 candidates and had each person read 66 or so, choosing the top 8 from their pool. Further say two people read each folder. Shouldn’t there be broad consensus across readers? Has anyone ever studied this? I’m speaking hypothetically here, but I bet it is common for there to be very little overlap between the two readers. If there were no overlap, we would end up with 48 people to take to the next level if everyone kept to choosing just 8.  I bet in most searches the number after a first cut like this is over 35, a surprising lack of consensus.

If this were science, we would go back to the beginning and say our process was not discovering what we want to discover. But we do not do so. There are a couple of reasons. First, we do not have the time. Second, maybe it is likely that the very best people each person reads will make it to the top. It will be interesting to see if the final candidates were among the few chosen by both readers. By the way, the two readers are typically randomly paired in all combinations in the best processes.

Who are we likely to miss at this stage? I think it is the people with big ideas who do not publish prolifically. Or it might be the people with big ideas and excellent work who focus, as so many do, on what they are asking rather than what they have figured out. But probably most of the people we miss are just random errors, no good reason to have chosen Hannah over Isabel.

For better or for worse, this process will narrow the field to 48 or fewer people chosen by someone and the whole committee will read those files and discuss them. Will this lessen arbitrariness? We can again see how much overlap there is when everyone is reading everything. The goal of the next step is to cut this list by half or more in a process that begins to consider fit, collegiality, and teaching.

This is the process I most often see. But there are no rules from my university as to the best way to do this. I wish we could be informed more by social scientists who might have studied this and can tell me how best to optimize outcome and fairness while not sacrificing undue faculty time.

Advertisements

About Joan E. Strassmann

Evolutionary biologist, studies social behavior in insects & microbes, interested in education, travel, birds, tropics, nature, food; biology professor at Washington University in St. Louis
This entry was posted in Jobs and tagged , , , , . Bookmark the permalink.

2 Responses to How scientific and fair can faculty hiring be?

  1. DS says:

    This is a wonderful post on an important topic. Well done. A (tongue in cheek question) Why let humans do this? We are full of bias. Maybe we should have the department that is hiring submit the CVs of the current faculty judged to be fully successful, add some criteria about research topic and other traits desired (e.g. a diverse top 8) and let an algorithm select the top 8 from the candidate pool. Match.com for the Academy. As a thought experiment, consider if, for your department, this would lead to the selection of those in the “ideal candidate pool” in the graph.

  2. We now separate the paperwork for male and female applicants and shortlist each sex separately in a bid to a) acknowledge bias and b) compare like with like. So far the process has yielded more women in the shortlist has the unforeseen effect of generating more diversity in the male shortlist. The downside is that we now have very long shortlists – because we don’t know how to condense the two shortlists. And we still have a very very low number of female faculty members. We need better and more proactive leadership to cross this very difficult hurdle.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s