Rubrics seem to be all the rage these days, whether they are appropriate or not. A rubric is simply a system for assigning points to different aspects of an assignment. They can be very useful for communicating to a student where they got points and where they did not. I have given a rubric for writing here. I have pointed out some of the problems with grading by rubric here. So isn’t using a rubric for choosing a new faculty member a great idea? Won’t it even avoid bias? After all, you could give extra points for minority status, or to women?
Here I argue that using a rubric for hiring is a terrible idea. Why? After all, we can all easily think of the elements that might go into a rubric. Here are a few: importance of the research area, research accomplished, independence, collegiality, teaching excellence, gender or minority status, and fit in the department. You might think of others. Yes, you might be thinking, those things are important, so why not weight them and pick the people with the highest points?
The problem comes from the very nature of the task of picking a new faculty member. Ultimately we will only hire one person from the more than 200 possibilities. A rubric evaluates all the characteristics at once. This inevitably devalues any given attribute, even if it is weighted higher than the others. Someone who is mediocre in research but fabulous in other characteristics could rise, for example.
What is a better process for choosing? It is a hierarchy of selection. Think of it as a series of successive sieves with the first one selecting only on the most important characteristic. What this is might differ among institutions or searches. In our searches for a tenure track faculty member, I think this first most crucial sieve should retain only the most excellent researchers. These people have chosen important research problems and are showing every indication of making fundamental advances. This is what CVs are about. This is what the letters talk about. This is what will get these people funding and international recognition. These are the people most likely to have a career of sustained excellence and interest. These are the people that will make our institution shine and get ranked highly. Anyone that is not the very best in research, a star or someone who gives every indication of being a rising star should simply not be chosen, no matter what their other characteristics are.
When you first select on this ground alone, the initial discussions by the committee will be about research. What is an important question? What exciting progress has a candidate made in a given area? What are new and important research areas? What areas are playing out? Where might the field be in a decade? In two decades? Is this person moving in an exciting direction? These are the fun conversations because they are about science, what we all love.
Out of our 200 candidates, this sole focus on research might yield a pool of ten or fifteen people. This selected pool might even be larger if the committee has diverse perspectives on research, but it is not likely to be much larger because we will educate each other in the discussion focused on science.
Only when we have this smaller pool of amazing researchers should we start worrying about balance, fit, collegiality, and teaching excellence. We might have more actual sieves, ruling out people that seem to be difficult, for example, but we need to be sure that we are not introducing bias at this level.
Yes, what about women and under-represented minorities? How do we get them in our final pool and on our faculty? I would argue it is not by giving them a point or two extra with a rubric. After all, that cannot overcome our very well documented tendencies towards unconscious bias. We have to think about that bias and its consequences at every step of the way, from the first screen of research excellence to the final screens that yield our interview slate and ultimate hiring decision. Take out that bias and women and under-represented minorities will naturally make up a big percentage of that final pool. After all, we have survived in an often unfriendly system because of our passion for research.
Oh, I hope I don’t even have to mention that there are a lot of things that should not be a part of the discussion at any stage. Some of these are illegal to consider. Others are pointless. After all, we do not have perfect information on them. So don’t talk about who might move. Don’t talk about the partner of the potential candidate. You get the idea.
Our task is challenging and vitally important for our department and our university. Let’s not forget what it is all about.