Back in Part 1 of this series on how to get a faculty job, I said there would be three parts in total. Well, I lied. I realized it would also be helpful to shed light on the process as seen by a faculty hiring committee, so in this post I'll augment Part 1 with a little behind-the-scenes of how hiring committees read and rank applications, and how interviews are granted. The "real" Part 2 will be about the interview itself, and Part 3 about negotiating the offer.
I served on the hiring committee at Harvard back in 2008 when we hired three great new Computer Science faculty: Krzysztof Gajos, Steve Chong, and Yiling Chen. It was an exhausting, months-long search with a dozen or so interviews for multiple openings (it had been a few years since we had any faculty openings and we really opened up the floodgates). So I have a little sense of how the sausage is made.
It's a complex process and utterly opaque for the poor applicant, who will usually not hear anything for many months after submitting the application. Most of the time, the response is a polite email from the hiring committee chair that because of the large number of highly qualified applicants for the position, they are very sorry that they will be unable to interview you. That is, if they ever contact you at all. Most schools don't bother even declining your application explicitly. You just never hear anything. (As for me, I'm still holding out hope that Stanford wants to interview me. It's only been 10 years since I sent my application, I guess it's still under consideration.)
Sometimes, though, you get lucky and are actually granted an interview. The most direct approach is an email saying that they are very interested in your application and would like to see if there are some dates you would be able to come for an interview. However, in some cases, a school doesn't want to "blow" one of its precious interview slots (more on that below) on an applicant who is not serious about their school. This will happen for a rock star candidate who is going to get interviews at MIT and Berkeley and only applied to your school to be polite, or as a backup. It would be a waste of time to interview such a candidate unless the department really feels it has a shot at landing this person. So, rather than directly offering an interview, the hiring committee might use side channels to find out if the applicant is serious about interviewing first -- for example, by getting in touch with the student's advisor and finding out more about what they're looking for in a school.
It's important to keep in mind is that whenever there is a faculty opening at any halfway-decent academic department, they will usually get inundated with hundreds or even thousands of applications from all corners of the globe. I am not exaggerating. The vast majority of these applicants are from schools you've never heard of in random countries where English is not the official language, and these people will rarely if ever get interviews (at least at good schools in the US).
The other thing is that most departments have only so much "interview bandwidth." Interviewing more than, say, a dozen applicants for a single position is very difficult. An interview typically lasts one or two days, you only have so many slots during the week in which to schedule job talks, and the committee has to spend a lot of time processing and discussing each interview. If a school has multiple openings in a year, they might still only interview a dozen or so candidates in total.
So, how do hiring committees decide who gets interviewed? There are about a million variables involved, but here are some of the most important:
Qualifications. Obviously this is important, but who counts as "qualified?" Your publication record is probably the strongest indicator of your success. Publishing at least one major conference paper a year -- after your first year or so in grad school -- is par for the course. Two or three papers would be a good year for most applicants. These have to be in good venues: Top-ranked, highly-competitive conferences. Workshops don't count (OK, maybe a little, but a lot less than real conferences). Journals don't count either. (This varies by field. In Computer Science, journals don't matter very much. But an article in Science or Nature will get you interviewed just about anywhere.)
Also, being first author on these papers is really important. Second author says, OK, maybe this student wasn't the most senior one on this piece of work, but they probably still contributed a lot. Third author on down conveys that you were not that involved and therefore get fewer points for the publication.
So you should expect to have something like five or six major conference publications -- ideally as first author -- on your CV, at minimum, to be taken seriously by most departments. Best paper awards are a big plus too, so list them on your CV whenever you get one. It is not uncommon these days to see applicants with ten or more top papers. I think this is a little insane. If you do a postdoc, though, you're expected to publish a good chunk of papers during that time, at minimum two a year -- the bar is higher for postdocs.
Your academic credentials matter a lot too. Your undergrad institution is not that much of a factor. I know plenty of famous faculty at top-10 schools who went to seemingly random undergrad institutions: Greg Morrisett, for example, apparently graduated from some place called the University of Richmond, which I'm sure is a very good school but is hardly a household name. What matters much more is where you went to grad school and (if you are doing a postdoc) where you postdoc. There is a reason that so many of the faculty at top-20 CS departments graduated from the likes of MIT, Berkeley, CMU, and Stanford -- graduates of these schools are highly sought after by CS departments and they tend to churn out enough graduates to fill the ranks of the top departments. So if you're coming from anything other than a top-20 school yourself, your chances of landing an interview at a higher-ranked institution are slim to none. (I know a bunch of people will argue with me here, and point out exceptions to the rule, but let's be honest. There is a strong preference for graduates of top-ranked departments when trying to pick 10 or so candidates to interview out of a pool of hundreds.)
The same goes if you're doing a postdoc. Actually, a postdoc can be a great way to increase your station in life if you didn't graduate from a name-brand department but still want a decent faculty job. Postdocing at MIT is almost (but not quite) as good as graduating from there.
The good news is that none of this shit matters if you do get an interview: No sane hiring committee is going to go back to your résumé and say, "Well, I really loved her interview, but she graduated from a lower ranked school than the other guy, so let's hire him instead." All of this is just about getting the interview. After that you're on your own.
Being a woman or a minority helps too. Hiring committees spend a lot of time trying to find anyone other than white men to interview, and most departments would love for their next hire to help increase the diversity of their faculty. This is a good thing, and is becoming more important as the diversity of the student population grows as well. If you happen to be one of these "underrepresented" candidates, more power to you -- given how competitive the academic job market is, you need every advantage you can get. (But see above about how this doesn't matter once you get the interview. That applies here too.)
Research area fit. This is a really complicated, multivariate function, in which the department attempts to discern, based on your CV, research statement, teaching statement, and letters, how well you would "mesh" into the department, whether you do the "kind of research" they are looking for, whether you can teach the classes that require coverage, and if you are likely to find collaborators in the department. It sounds like a lot to worry about, but the answer for you, as an applicant, is simple: It's too late for you to do anything about this. If you're a sixth-year PhD student applying for faculty jobs, it's too late to "rebrand" yourself to try to optimize for some complex, black-box process that is going on within each of the departments you're applying to. The time to figure out what research problems are going to look sexy on a job application is when you're a first or second year grad student, but, by the time you graduate those problems are just as likely not to be sexy anymore -- so instead, just do the research you love and hope you find a department that is looking for someone like you.
Sometimes you don't get an interview due to factors totally beyond your control. For example, I didn't get interviewed by a couple of departments because they had just recently (in the last year or so) hired graduates of my same research group at Berkeley. That poisoned the well for me -- there was no way I could pretend to not be doing research in the same area under the same set of professors. (There are still times I shake my fist at the night sky and scream "Armandooooooooooooo!")
Finally, your recommendation letters are key. I could write an entire blog post about what a good faculty recommendation letter should say, but you as a job applicant have little control over what your letters will look like. The letters touch on many things: Your technical and intellectual capacity, your research taste, your teaching style, your personality, your chances at getting tenure down the road. As an applicant, what you can do is make sure you talk to your letter writers before they write the letter. This is for several reasons. First, you want to address any questions or concerns they have about you up front. For example, there might be some lingering questions about how much you contributed to some project a few years back, and talking about it openly with your references gives you a chance to clear up any confusion. Also, your reference needs to understand what you're looking for in a faculty job. Say you are applying to a mix of top-ranked research universities and a few smaller teaching schools. This can lead to confusion: What kind of job are you after? Do you want to mostly teach? Or are the teaching schools a safety net? You need to give your references a chance to ask these questions directly rather than infer the wrong thing and write a blind letter.
What's the process like for the hiring committee? Usually, the committee will meet several times, go through the applicants, rank them in various ways, and try to reach consensus on whom to invite for interviews. This can take a month or more. At first, a couple of interviews might be given out to the clear front-runner candidates that they really want to snag early (since good candidates' interview schedules fill up too). Then a few more weeks of deliberation happens while the rest of the interviews are sorted out. Keep this in mind: If you haven't heard from a school, but know they have started scheduling interviews (say, by looking at their online events calendar where it's usually pretty obvious who's giving a job talk), that may not mean that all of the interviews have been decided yet: it's usually a rolling process. Generally the first interviews start to get scheduled around February, but March and April is when things really get going.
In the next part I'll talk about how to nail the interview.
I served on the hiring committee at Harvard back in 2008 when we hired three great new Computer Science faculty: Krzysztof Gajos, Steve Chong, and Yiling Chen. It was an exhausting, months-long search with a dozen or so interviews for multiple openings (it had been a few years since we had any faculty openings and we really opened up the floodgates). So I have a little sense of how the sausage is made.
It's a complex process and utterly opaque for the poor applicant, who will usually not hear anything for many months after submitting the application. Most of the time, the response is a polite email from the hiring committee chair that because of the large number of highly qualified applicants for the position, they are very sorry that they will be unable to interview you. That is, if they ever contact you at all. Most schools don't bother even declining your application explicitly. You just never hear anything. (As for me, I'm still holding out hope that Stanford wants to interview me. It's only been 10 years since I sent my application, I guess it's still under consideration.)
Sometimes, though, you get lucky and are actually granted an interview. The most direct approach is an email saying that they are very interested in your application and would like to see if there are some dates you would be able to come for an interview. However, in some cases, a school doesn't want to "blow" one of its precious interview slots (more on that below) on an applicant who is not serious about their school. This will happen for a rock star candidate who is going to get interviews at MIT and Berkeley and only applied to your school to be polite, or as a backup. It would be a waste of time to interview such a candidate unless the department really feels it has a shot at landing this person. So, rather than directly offering an interview, the hiring committee might use side channels to find out if the applicant is serious about interviewing first -- for example, by getting in touch with the student's advisor and finding out more about what they're looking for in a school.
It's important to keep in mind is that whenever there is a faculty opening at any halfway-decent academic department, they will usually get inundated with hundreds or even thousands of applications from all corners of the globe. I am not exaggerating. The vast majority of these applicants are from schools you've never heard of in random countries where English is not the official language, and these people will rarely if ever get interviews (at least at good schools in the US).
The other thing is that most departments have only so much "interview bandwidth." Interviewing more than, say, a dozen applicants for a single position is very difficult. An interview typically lasts one or two days, you only have so many slots during the week in which to schedule job talks, and the committee has to spend a lot of time processing and discussing each interview. If a school has multiple openings in a year, they might still only interview a dozen or so candidates in total.
So, how do hiring committees decide who gets interviewed? There are about a million variables involved, but here are some of the most important:
Qualifications. Obviously this is important, but who counts as "qualified?" Your publication record is probably the strongest indicator of your success. Publishing at least one major conference paper a year -- after your first year or so in grad school -- is par for the course. Two or three papers would be a good year for most applicants. These have to be in good venues: Top-ranked, highly-competitive conferences. Workshops don't count (OK, maybe a little, but a lot less than real conferences). Journals don't count either. (This varies by field. In Computer Science, journals don't matter very much. But an article in Science or Nature will get you interviewed just about anywhere.)
Also, being first author on these papers is really important. Second author says, OK, maybe this student wasn't the most senior one on this piece of work, but they probably still contributed a lot. Third author on down conveys that you were not that involved and therefore get fewer points for the publication.
So you should expect to have something like five or six major conference publications -- ideally as first author -- on your CV, at minimum, to be taken seriously by most departments. Best paper awards are a big plus too, so list them on your CV whenever you get one. It is not uncommon these days to see applicants with ten or more top papers. I think this is a little insane. If you do a postdoc, though, you're expected to publish a good chunk of papers during that time, at minimum two a year -- the bar is higher for postdocs.
Your academic credentials matter a lot too. Your undergrad institution is not that much of a factor. I know plenty of famous faculty at top-10 schools who went to seemingly random undergrad institutions: Greg Morrisett, for example, apparently graduated from some place called the University of Richmond, which I'm sure is a very good school but is hardly a household name. What matters much more is where you went to grad school and (if you are doing a postdoc) where you postdoc. There is a reason that so many of the faculty at top-20 CS departments graduated from the likes of MIT, Berkeley, CMU, and Stanford -- graduates of these schools are highly sought after by CS departments and they tend to churn out enough graduates to fill the ranks of the top departments. So if you're coming from anything other than a top-20 school yourself, your chances of landing an interview at a higher-ranked institution are slim to none. (I know a bunch of people will argue with me here, and point out exceptions to the rule, but let's be honest. There is a strong preference for graduates of top-ranked departments when trying to pick 10 or so candidates to interview out of a pool of hundreds.)
The same goes if you're doing a postdoc. Actually, a postdoc can be a great way to increase your station in life if you didn't graduate from a name-brand department but still want a decent faculty job. Postdocing at MIT is almost (but not quite) as good as graduating from there.
The good news is that none of this shit matters if you do get an interview: No sane hiring committee is going to go back to your résumé and say, "Well, I really loved her interview, but she graduated from a lower ranked school than the other guy, so let's hire him instead." All of this is just about getting the interview. After that you're on your own.
Being a woman or a minority helps too. Hiring committees spend a lot of time trying to find anyone other than white men to interview, and most departments would love for their next hire to help increase the diversity of their faculty. This is a good thing, and is becoming more important as the diversity of the student population grows as well. If you happen to be one of these "underrepresented" candidates, more power to you -- given how competitive the academic job market is, you need every advantage you can get. (But see above about how this doesn't matter once you get the interview. That applies here too.)
Research area fit. This is a really complicated, multivariate function, in which the department attempts to discern, based on your CV, research statement, teaching statement, and letters, how well you would "mesh" into the department, whether you do the "kind of research" they are looking for, whether you can teach the classes that require coverage, and if you are likely to find collaborators in the department. It sounds like a lot to worry about, but the answer for you, as an applicant, is simple: It's too late for you to do anything about this. If you're a sixth-year PhD student applying for faculty jobs, it's too late to "rebrand" yourself to try to optimize for some complex, black-box process that is going on within each of the departments you're applying to. The time to figure out what research problems are going to look sexy on a job application is when you're a first or second year grad student, but, by the time you graduate those problems are just as likely not to be sexy anymore -- so instead, just do the research you love and hope you find a department that is looking for someone like you.
Sometimes you don't get an interview due to factors totally beyond your control. For example, I didn't get interviewed by a couple of departments because they had just recently (in the last year or so) hired graduates of my same research group at Berkeley. That poisoned the well for me -- there was no way I could pretend to not be doing research in the same area under the same set of professors. (There are still times I shake my fist at the night sky and scream "Armandooooooooooooo!")
Finally, your recommendation letters are key. I could write an entire blog post about what a good faculty recommendation letter should say, but you as a job applicant have little control over what your letters will look like. The letters touch on many things: Your technical and intellectual capacity, your research taste, your teaching style, your personality, your chances at getting tenure down the road. As an applicant, what you can do is make sure you talk to your letter writers before they write the letter. This is for several reasons. First, you want to address any questions or concerns they have about you up front. For example, there might be some lingering questions about how much you contributed to some project a few years back, and talking about it openly with your references gives you a chance to clear up any confusion. Also, your reference needs to understand what you're looking for in a faculty job. Say you are applying to a mix of top-ranked research universities and a few smaller teaching schools. This can lead to confusion: What kind of job are you after? Do you want to mostly teach? Or are the teaching schools a safety net? You need to give your references a chance to ask these questions directly rather than infer the wrong thing and write a blind letter.
What's the process like for the hiring committee? Usually, the committee will meet several times, go through the applicants, rank them in various ways, and try to reach consensus on whom to invite for interviews. This can take a month or more. At first, a couple of interviews might be given out to the clear front-runner candidates that they really want to snag early (since good candidates' interview schedules fill up too). Then a few more weeks of deliberation happens while the rest of the interviews are sorted out. Keep this in mind: If you haven't heard from a school, but know they have started scheduling interviews (say, by looking at their online events calendar where it's usually pretty obvious who's giving a job talk), that may not mean that all of the interviews have been decided yet: it's usually a rolling process. Generally the first interviews start to get scheduled around February, but March and April is when things really get going.
In the next part I'll talk about how to nail the interview.
Hey Matt, one minor correction. In some areas of CS, such as theory, paper authors are listed alphabetically. Others, like bioinformatics, adopted the biomedical convention First Author > Last Author > Authors in the Middle (although this convention itself is disputable, because the main contribution of the PI/Last Author might have been to run the lab...). Clueful hiring committees must be aware of these differences when reading the tealeaves in publication lists.
ReplyDelete"There are still times I shake my fist at the night sky and scream "Armandooooooooooooo!"", haha
ReplyDeleteAnother small correction: ""This is a good thing, and is becoming more important as the diversity of the student population grows as..""
ReplyDeleteNo, this is a bad thing. Bigotry in any form is bad. It is bigotry to use the race/color/gender in any manner during the hiring process - even to give underrepresented groups an edge. The hiring committee should be color-blind.
Thanks for writing these posts!
ReplyDeleteI think that as a post-doc, being first author isn't as critical as for a graduate student, but in all cases, it is critical for your letter writers to explain your contribution to the most highly visible pieces of your work.
I've been on the faculty at CMU and Berkeley for 27 years, and I cannot recall a single time that author order ever came into the discussion of recruiting or promotion.
ReplyDeleteSo I'd like to echo Doug's comment. I've also never seen a discussion of paper order at UCSD. Moreover, I've never seen a discussion of school. As a rule we _do_ care about who your letter writers are and for some areas you'll have better access to letter writers at some schools... but that's really the issue, not brand per se. I also think that your readers may not understand from your post that there is a very real and significant hierarchy among the various signals that drive interview invite decisions. For example, by far the number one mechanism by which candidates are selected is driven by an advocate in the institution. It is very common for faculty at good schools to keep track of who the strong graduating students are in their area and to let the recruiting committee know who they think the top candidates is. This is not sufficient (i.e., you can't have terrible letters and no publications) but it is a huge advantage and this is why networking is valuable for students. I'd say up to 50 percent of candidates come through this path. To put it another way, we already know about roughly half of the candidates we interview. Next are the candidates who are personally unknown, but have strong letters. This requires some institutional memory to understand how to interpret particular letter writer's prose (i.e., what does "in the same equivalence class as X" really mean from author Y). However, these are hugely important since they provide a reputational shorthand for interpreting a career and its importance. I'd say this brings in another third of candidates. Finally, we come to the publication record. I say "finally" because this is important, but is simply not the most important signal in the process. I've seen very little "bean counting" where candidates are ranked based on number of publications. Having ten publications in good venues, by itself, is not a particularly easy way to get an interview (hopefully, in the process of doing all those publications you've managed to convince others to vouch for you -- either at the institution you're apply to or from your letter writers). Conversely, there have been candidates with far more limited publications histories who were very successful (e.g., Dina Katabi was not terribly prolific as a graduate student, but was broadly known in the networking community for her work -- both published and unpublished -- and her letters were very strong).
ReplyDeleteI'd like to second Stefan on the "advocate in the institution" thing. I work in an industry research lab, not at a university, but our hiring processes operate under similar constraints (limited slots, too many applications). If I've met a student at a conference, ideally someone who has given a clear and interesting talk, and who also then talk intelligently 1-on-1 about his/her own work AND about other things, I'm going to tell my colleagues that this person should be high on our list of candidates to invite for an interview. It won't be the main deciding factor, of course, but making a good impression as an intelligent person can compensate for having slightly fewer top-conference papers. So, students: go to conferences, give good talks, and get introduced to people who might end up advocating for you some day.
ReplyDeleteVery true about having an advocate, good point Stefan.
ReplyDeleteI didn't mean to suggest that number of papers and author order is ever explicitly "scored", but I'd be amazed if this were not at least in the back of people's minds when reading over an unfamiliar resume. If I had a resume with one or two conference pubs where the student was not the first author, I'd question whether this was the strongest person we could be interviewing. Of course other factors (letters, reputation, etc.) can override such a concern, but I don't think it would be good advice to grad students looking for faculty jobs to ignore their publication record.
Yeah, I think we may just disagree on this one. I think focusing on author order actually creates a perverse incentive among students to not collaborate because that will be time during which they will not be collecting a "first-authored" paper. In my experience such students end up being much less productive and much less attractive as colleagues. Particularly in areas where group projects are the norm its inevitable that you can't bee first author all the time. I think the key issue is whether you made a significant contribution to the work and that your letter writers can explain that. Now its certainly weird if you're not the first author on _anything_ but I'd always tell students to focus on doing interesting work rather than author order. Generally, your advisor should be looking out for you and will help communicate what you did (now if you don't trust your advisor then you have a bigger problem).
ReplyDeleteSimilarly, I think the drive to have lots of publications can bee self-defeating. Quantity is not quality and even at high-quality venues there are lots of papers that no one really remembers very well. Far preferable to publish less and work on something memorable.
When you interview you're whole career gets (unfairly) summed up very concisely. She's the "Using compilers for privacy" woman, he's the "power management for NICs" guy, etc. I think its far easier to cut through if you did one or two really interesting things than to be "he's the guy with 6 SIGCOMM papers"
I totally agree that students shouldn't care about author order, or even publication count for that matter -- but as long as hiring committees care about these things, then students should too. I guess you might be arguing that hiring committees care less about this stuff than I believe they do, but my data (sitting on hiring committees and paying close attention to who gets job offers where for many years) suggests otherwise: This stuff does matter, a lot. Like you I wish that were not the case.
ReplyDeleteFrom the more theoretical side, I'll echo that to me author order doesn't matter, because in theory our default is alphabetical order. I've worked with people in other areas, though, where author order has deeply significant meaning that I try hard not to understand. In CS, it's generally not an issue at hiring time; I can't recall it coming up in any significant way.
ReplyDeleteI'd also echo Stefan and Jeff in their idea that networking BEFORE you're doing the job search is huge. Here I mean networking in the general sense of making yourself known to the community -- things like going to conferences, giving talks, even doing reviews for people. Also, I'd strongly highlight doing a summer internship or doing something else away from your grad institution as an important aspect of networking. It's sort of obvious -- it introduces you to a whole new group of people -- but some students don't seem to get that. [Note/e.g.: my first job out of grad school was at DEC SRC, where I had done a summer internship my 3rd year...] While I am enjoying Matt's very informative posts, I think you can't emphasize enough that if you haven't put in the legwork throughout your graduate career -- and that legwork is more than just writing your research papers -- you're at a very severe disadvantage at hiring time.
Finally, I'll just keep echoing others and say Stefan's point about quantity is not quality is very important. Yes, the committee probably wants to see some breadth in your ability. But as Stefan points out, you'll be summarized (or, perhaps better said, "compressed") to your most important and interesting ideas and characteristics. I should point out that that's not just at the interview, but will happen to you throughout your career. I'd wager that of the CS community that knows my work, 90+% think of me as "the balls and bin guy, does hashing and Bloom filters, will work with systems people". I might complain that as Stefan describes that's an "unfair" summary, but actually I'd have to admit, it's worked out well for me.
Maybe its a Harvard thing... :-) Seriously, I've been doing faculty recruiting for a bit over a dozen years now, and probably with a more quantitative focus than most (e.g., we maintain lists of who interviews at all of the top 30 or so schools) and I think you're simply wrong or confusing correlation with causation. For example, look at MIT systems/networking hires over the last decade. Nikolai, Dina, Robert... none of them had big publication histories. To put it another way, there were quite a few candidates with far more publications than they had at the time, but who did less well. Look, this isn't to say that being prolific hurts you, its just not as important as you're making it out. It so happens that there are people who manage to do great work and publish a lot and they do fine too... but its not the volume that won them the attention. Again, in the end the main reason you get interviews is institutional knowledge or letters -- not your CV -- and ultimately you can't be known for 8 different things. You get known for one or two things and those have to be memorable. If you wrote a bunch of other non-great papers in good venues, that's fine too, but its icing. Every year I see candidate files with tons of publications (e.g., eight or more papers in top venues such as NSDI, SIGCOMM, OSDI, etc) who do not do well on the market because their work simply didn't excite anyone.
ReplyDeleteJust saw Mike's note... I guess its not a Harvard thing :-)
ReplyDeleteI think you've misinterpreted my post as saying "more papers are always better", or that quantity matters more than quality, or that publication count is all that matters. Of course not. Doing high quality work and being well known are far more important. So I completely agree with that.
ReplyDeleteAll I'm saying is that there is probably a minimum bar for a candidate's publication record to be taken seriously by most hiring committees. You don't need 10 or 20 papers - but you probably need more than 1 or 2. I recall when I was on the committee at Harvard that we had one candidate with 2x the number of papers of everyone else we interviewed, but we recognized this person's output was somewhat lower quality and more scattered than the more focused candidates. More is not always better.
But I don't think anyone can disagree that most people judge research impact directly or indirectly by some function of both quality and number of publications. If that were not true then we wouldn't care about papers at all.
On the other hand ... I was recently at a Dagstuhl workshop on "Publication Culture in Computing Research" and was somewhat horrified to learn that there are CS departments where the faculty expect an administrative assistant to make up a table of applicants that includes the number of top-tier publications for each applicant ... and then people only read the CVs of the applicants with the most papers. Sorry, I can't remember who said this, but my recollection is that this was not viewed as entirely abnormal, although pretty much everyone at the workshop said "of course, in my department we would never do such a thing."
ReplyDeleteMy sense was that top departments become top departments partly because they look deeper than just publication counts ... but perhaps some not-so-top departments aren't so careful.
The reason to write papers is to tell people about something interesting you did, something interesting you learned and why its all important. The papers are the story form by which we relate the interesting results from the research process. I think the problem we've gotten into is that increasingly there is a sense that we should be writing papers for their own sake. I don't think this is very useful.
ReplyDeleteOn an unrelated note, two things for candidates to keep in mind that haven't been mentioned yet: first, since just about all recruiting is done on a rolling basis, there is inherent positional priority. Hence, its a good practice to get after your letter writers come the first or second week in January to make sure they're on the ball. There are a finite number of interview slots and a small set still in your area. Second, is the somewhat controversial issue I call letter dynamic range. Very few of your letter writers are going to write multiple different letters for you. Consequently, that letter (which is backed only by the writer's finite reputation) is going to need to cover the full range of institutions you apply to. If you end up applying to places that your letter writer does not believe you are competitive for then this is going to lead to -- for want of a better word -- letter compression; your letter will not be as strong since it is implicitly an assertion of whether the writer believes that the recipient should interview you. This is why its worth having a frank discussion with your letter writers ahead of time about where you and they think you are competitive.
DEC SRC had similar hiring criteria as academic departments but a few luxuries including (a) a director that drilled into us that hiring was the most important thing we did, (b) less administrative overhead, so more time could be devoted to the process, and (c) long term stable staff and long term relationships with a wide range of academic faculty and staff at other research institutions.
ReplyDeleteAs Michael hinted, hiring was often a multi-year process - promising candidates were often brought to our attention by their advisors and tracked through their graduate careers via a number of channels including meeting at conferences, internships, and occasional collaboration with SRCers. I agree with Stefan, just having a paper wasn't important - but having a paper/talk that demonstrated interesting learning and insight was crucial. These sorts of candidates were well known quantities by the time they were interviewed and their publication record and references were secondary.
This wasn't the only way in. We were encouraged to "cast our net broadly" to make sure we didn't just hire "people that looked just like us", and here publications and letters were paramount. One of the criterium was how well "calibrated" we were on the letter writer. Occasionally we would invite someone in mostly on the strength of a single very strong letter with the objective of calibration of the letter writer as much as the candidate. (When Don Knuth wrote that one of his students "reminded him of a young Gauss" we didn't bother soliciting more material. Today we probably would have immediately dispatched a limo to fetch him, but those were different days.) This process lead to significant institutional memory and we were fairly successful filtering the number of candidates to a bandwidth we could handle with very few people we didn't feel were worth the interview time.
Finally, having an advocate was required - unlike academia our work was essentially collaborative. It was possible for one champion to bring a candidate in and another (or others) to arise through the process. Our view was that the Center was responsible for making a new hire successful and this was not possible without at least one SRCer taking personal responsibility. This was a huge difference from what I've observed in academia.
So for a grad student (a) have good ideas you can write up, (b) have an advisor with a good reputation that is willing to look out for you, (c) use conferences as opportunities to meet prospective hiring committees.
I wonder if this reframing of the question (who should get an interview?) will generate any useful observations: It seems to me that people tend to focus on what qualities make a candidate clearly stand out from the pack. But I think it's relatively easy to agree on who those people are, no matter whether you put more weight on pedigree or the Nth high value publication or the Mth glowing letter of reference. I assume that outside of the few most competitive departments and labs it's not feasible to insist on only interviewing obviously outstanding candidates.
ReplyDeleteSo the hard question is once you get back into the pack a little bit, what are you willing to compromise on first? Publication count? Brilliance of the candidate's best idea? Perceived collegiality? Maybe one useful way of measuring this is what objections by members of hiring committees tend to sink candidates versus what objections tend to get overridden?
As a postdoc, who is currently looking for academic/lab jobs, it seems somewhat scary that author order matters in publications. All my graduate student and postdoc life, I've always thought that even if you have done the maximum work on a paper, even if it is a paper based on your thesis work, courtesy demands that you put others' names before yours. Especially in two-author papers -- if there are only 2 authors, well someone's name had to go first!
ReplyDeleteIt seems somewhat impolite to insist that one's name goes first, and such insistence could sour or mar otherwise good working relationships that might have taken years to build.
Anonymous #1 (7:58am) The dynamics by which schools modify their evaluation based on their sense of their own competitiveness is complex. Unfortunately, among the top 30 or so schools there is tremendous drive for self-improvement and hence what you find is that the same 3-4 people in a given area will get 70+% of the interviews. This obviously depends on external factors (e.g., demand for particular areas, impact of particular personalities in recruiting committees, etc) but the winner-takes-all model tends to hold pretty well. There is also a tier of schools (perhaps 20-50) who collectively define a second tier of candidates for a given area, but the spread here is much larger and more random (i.e., its still the case that a smaller number of applicants than positions eat up most of the interviews, but its not just 3-4... its maybe 5-8 candidates). I'd be hard pressed to say that there is one singular metric that dominates above the knowledge > letters > publications model I discussed earlier. However, as a rule the less the recruiter knows about the area, the more publications can dominate (since you don't trust your own knowledge of the area or your ability to rate letters). Thus, in departments where the Dean micromanages or holds particular sway, there can be a tendency to focus on more empirical quantifiable metrics (publications, awards, citations, etc). This is also why such metrics appear around promotion time -- because outside evaluators do not trust themselves to do the evaluation fairly and are desperate for external metrics to drive their opinions. Again, the best thing that can be done is for someone in the organization to already have a positive opinion about you.
ReplyDeleteAnonymouys #2 (10:44). I think your attitude is very nice, but perhaps to the point of letting yourself be taken advantage of. In fields in which author order is indicative of some special merit, when you have not received your PhD yet and when you have contributed the plurality of the work on an effort, it is appropriate that your name should be first (there are some counter-indications which can relate to how the project was started, etc, but lets ignore those). However, it is unfortunate in this case if you are left to make this decision yourself. Indeed, I think your intuition that pushing to have your name in a particular place can mar relationships. Generally, your advisor/mentor should be handling this issue and doing the appropriate thing. Now in some fields it is common that post-PhD contributors put their name at the end of the paper instead... and as Michael explains in other fields they just do alphabetical order. This is again why its a bad idea to read too much into author order in lieu of some other mechanism for understanding contribution. Hopefully, your advisor and senior collaborators understand the community in which you're working and do the right thing by you.
Really want to read your next part!! Your blogs are very helpful to me, who targets a faculty job for next year.
ReplyDeleteWhat matters much more is where you went to grad school and (if you are doing a postdoc) where you postdoc. There is a reason that so many of the faculty at top-20 CS departments graduated from the likes of MIT, Berkeley, CMU, and Stanford -- graduates of these schools are highly sought after by CS departments and they tend to churn out enough graduates to fill the ranks of the top departments.
ReplyDeleteWe looked into this a while back. The big four (CMU, Stanford, Berkeley MIT) seemed to have a rather eclectic collection of graduates among their professorial ranks.
The next 10 departments did seem to have a rather marked preference for big four graduates. Yes, even Harvard and Princeton got short changed in that "quest for graduate from big department" policy.
However, at my home institution (top 20) our experience is that the big four graduates that we have a hope or landing are not nearly as good as the best candidates we can land from "lesser" institutions, where by lesser we mean Cornell, Princeton, UW, Harvard, Duke, Maryland and other top 30 or so departments.
Once you are outside those, you better walk on water before we even consider bringing you from an interview, since often "best graduate we ever had" from a small institution is still likely not good enough to meet our tenure standard six years later if you were to be hired (we learned this the hard way over many years).
Anon re: "Looked into this a while back." I would love to know what institution you are at. I completely agree that the best students from non-top-four schools are going to dominate the second-tier students from the top four. Most places interview pretty broadly. It doesn't really matter which school you're from as long as you have a strong publication record: Nobody ever said, "Well, you're only from Harvard, so your publications don't count as much." In fact, some non-top-four places (like Univ. Washington and UCSD) are handily beating their top-four counterparts in specific areas.
ReplyDeleteIn fact, some non-top-four places (like Univ. Washington) are handily beating their top-four counterparts in specific areas.
ReplyDeleteIronically, UW being one of the places that seems to have the most hires from the top four among their ranks.
"Well, you're only from Harvard, so your publications don't count as much."
Of course not, but I have certainly heard first hand of places that almost automatically interview candidates from the big four while being much more selective for candidates from the next tier, forgetting all along that the best candidates from the next tier compare quite favorably with the middle rank from the big four.
Great post overall, but I would like to correct one issue:
ReplyDeleteYou categorically state that journals do not matter in CS. In fact, journals do matter a lot in some CS areas. For example, in machine learning a leading journal paper (JMLR) is much more significant than a leading conference paper (ICML/NIPS). This is also true in other areas that are not traditional CS but many CS students/faculty work at (vision, information theory).
Regarding ordering of authors, this depends on the area: in machine learning for example, in does matter a lot and I've seen this issue raised frequently in hiring committee discussions for machine learning candidates (excluding people from COLT community).