The Secret Lives of Professors

I came to Harvard 7 years ago with a fairly romantic notion of what it meant to be a professor -- I imagined unstructured days spent mentoring students over long cups of coffee, strolling through the verdant campus, writing code, pondering the infinite. I never really considered doing anything else. At Berkeley, the reigning belief was that the best and brightest students went on to be professors, and the rest went to industry -- and I wanted to be one of those elite. Now that I have students that harbor their own rosy dreams of academic life, I thought it would be useful to reflect on what being a professor is really like. It is certainly not for everybody. It remains to be seen if it is even for me.

To be sure, there are some great things about this job. To first approximation you are your own boss, and even when it comes to teaching you typically have a tremendous amount of freedom. It has often been said that being a prof is like running your own startup -- you have to hire the staff (the students), raise the money (grant proposals), and of course come up with the big ideas and execute on them. But you also have to do a lot of marketing (writing papers and giving talks), and sit on a gazillion stupid committees that eat up your time. This post is mostly for grad students who think they want to be profs one day. A few surprises and lessons from my time in the job...

Show me the money. The biggest surprise is how much time I have to spend getting funding for my research. Although it varies a lot, I guess that I spent about 40% of my time chasing after funding, either directly (writing grant proposals) or indirectly (visiting companies, giving talks, building relationships). It is a huge investment of time that does not always contribute directly to your research agenda -- just something you have to do to keep the wheels turning. To do systems research you need a lot of funding -- at my peak I've had 8 Ph.D. students, 2 postdocs, and a small army of undergrads all working in my group. Here at Harvard, I don't have any colleagues working directly in my area, so I haven't been able to spread the fundraising load around very much. (Though huge props to Rob and Gu for getting us that $10M for RoboBees!) These days, funding rates are abysmal: less than 10% for some NSF programs, and the decision on a proposal is often arbitrary. And personally, I stink at writing proposals. I've had around 25 NSF proposals declined and only about 6 funded. My batting average for papers is much, much better. So, I can't let any potential source of funding slip past me.

Must... work... harder. Another lesson is that a prof's job is never done. It's hard to ever call it a day and enjoy your "free time," since you can always be working on another paper, another proposal, sitting on another program committee, whatever. For years I would leave the office in the evening and sit down at my laptop to keep working as soon as I got home. I've heard a lot of advice on setting limits, but the biggest predictor of success as a junior faculty member is how much of your life you are willing to sacrifice. I have never worked harder than I have in the last 7 years. The sad thing is that so much of the work is for naught -- I can't count how many hours I've sunk into meetings with companies that led nowhere, or writing proposals that never got funded. The idea that you get tenure and sit back and relax is not quite accurate -- most of the tenured faculty I know here work even harder than I do, and they spend more of their time on stuff that has little to do with research.

Your time is not your own. Most of my days are spent in an endless string of meetings. I find almost no time to do any hacking anymore, which is sad considering this is why I became a computer scientist. When I do have some free time in my office it is often spent catching up on email, paper reviews, random paperwork that piles up when you're not looking. I have to delegate all the fun and interesting problems to my students. They don't know how good they have it!

Students are the coin of the realm. David Patterson once said this and I now know it to be true. The main reason to be an academic is not to crank out papers or to raise a ton of money but to train the next generation. I love working with students and this is absolutely the best part of my job. Getting in front of a classroom of 80 students and explaining how virtual memory works never ceases to be thrilling. I have tried to mentor my grad students, though in reality I have learned more from them than they will ever learn from me. My favorite thing is getting undergrads involved in research, which is how I got started on this path as a sophomore at Cornell, when Dan Huttenlocher took a chance on this long-haired crazy kid who skipped his class a lot. So I try to give back.

Of course, my approach to being a prof is probably not typical. I know faculty who spend a lot more time in the lab and a lot less time doing management than I do. So there are lots of ways to approach the job -- but it certainly was not what I expected when I came out of grad school.

Update 4/24/10: Thanks to Mike Belfrage for pointing me to this interview with Niklaus Wirth where he echoes some of the above sentiments.

Proposal: Improving the NSF Review Process

Summary: Let's improve the NSF proposal review process by making it function more like conference program committees.

Intellectual merit: The core problem that this proposal addresses is the poor quality of many reviews submitted by NSF panelists. It is not uncommon for a proposal to be rejected with short, content-free reviews, offering little feedback to the authors. In many cases the scoring of a proposal is poorly justified, leaving the author mystified as to why they got such a low (or high) score. Recently, I had a proposal rejected where one of the reviews was essentially a single sentence in length. Not only does this not help the PI improve the work for later submission, but it leaves the impression that the review process is arbitrary.

(I'd like to emphasize that this is a problem that many NSF program managers have called attention to, but they are powerless as individuals to do much about it. So I believe the fault rests with the research community, not with the NSF PMs.)

A key problem with NSF panels is that there is no community standard for what constitutes a good (or even acceptable) proposal review. I am a strong advocate of the approach used by the systems community, where paper submissions are given extremely detailed reviews with constructive feedback. Given that we spend so much effort reviewing papers, couldn't we also give the same effort to NSF proposals, which arguably are more important than a single paper?

It is my impression that NSF program managers also have a hard time pulling panels together, mainly because people are so busy, and don't have the time to travel to DC. Yet many of the potential panelists freely serve on conference program committees with much higher reviewing loads and an expectation of much more detailed reviews. (A typical panelist will review 8-12 proposals, whereas a competitive conference will require TPC members to review 2-3x as many papers.) Why? One reason, perhaps, is that program committees are recognized for their work, and serving on a TPC is an important indication of one's stature in the research community.

These two issues are related. Since serving on an NSF panel is seen as "paying your dues," rather than an activity you take pride in, there is little incentive to write good reviews. However, if you write a bunch of crappy reviews for a TPC, you can earn a reputation as someone who doesn't take the process seriously might not get invited back in the future. So the public recognition of the TPC and the quality of the reviews go hand in hand.

My proposal: Let's have NSF panels emulate the conference program committee model. First, we should recognize panelists publicly for their work. Being on the "NSF NeTS 2010" panel should be as prestigious as serving on SIGCOMM or SenSys. The NSF should create a web page for each years' competition where they list the panelists and list the proposals funded through that competition (the latter information is available, but a little hard to dig up). So the community can take pride in the effort and see the outcome of the process more directly.

Second, establish expectations for high-quality proposal reviews. If you are a bad panelist, you won't get invited back in the future, so you won't gain the recognition of being asked to serve. Panelists will be chosen from among the best people in the field, where "best" is defined both by research contribution and service.

Third, hold panels somewhere other than Washington DC. Since I live in Boston, it's easy for me to get down there, but for people on the West Coast it is much harder. If panels are run in different locations around the country, the travel burden can be spread around more evenly.

I will be the first to admit that the conference program committee model is not perfect -- see my related posts here and here for thoughts on that. But in my experience it is better than the (typical) NSF panel.

Of course, NSF's conflict-of-interest guidelines will have to be tweaked. Currently, you can't serve on an NSF panel for a program for which you have submitted a proposal. (The upshot is that panels tend to consist of people who didn't get their act together to submit a proposal, which may not be the best group of scholars to evaluate the work.) Recently NSF has separated larger programs into multiple panels and simply ensured that someone can't serve on the specific panel for which their own proposal is under consideration.

Broader impacts: Improving the proposal review process and providing more constructive feedback to PIs will result in better science.

In the comments, please indicate your rating for this proposal:
   Excellent
   Very Good
   Good
   Fair
   Poor

Geoff Challen and IDEA

First of all, I'm very pleased to report that my student Geoffrey Werner Challen (formerly known as Geoffrey Werner-Allen, for reasons explained on his own web page) just defended his thesis! Geoff was the lead student on our work on sensor nets for volcano monitoring and developed the Harvard MoteLab testbed (used by dozens of research groups around the world). His work on Lance and IDEA defined new paradigms for resource management in sensor networks. He will be joining the faculty at the University of Buffalo next year and I am very proud of him. Congrats!

Second, I just posted the final version of our MobiSys'10 paper on Integrated Distributed Energy Awareness for Wireless Sensor Networks. This paper deals with the problem of configuring individual nodes in a sensor net to achieve some network-wide energy objective, such as targeting a given network lifetime. Each node generally has many local parameters that can impact overall energy consumption, such as the choice of parent in a routing tree or the MAC protocol duty cycle. We observe that nodes performing a greedy, local decision will often get it wrong; for example, a node deciding to forward packets through a parent with limited energy can rapidly drain the parent's supply. When energy harvesting (such as solar charging) is employed, it becomes very difficult to tune each node to achieve a network-wide objective.

The idea behind IDEA (sorry!) is to enable individual sensor nodes to make decentralized decisions about how to configure their own state. Nodes share information on their own battery level, charging rate, and energy load with other nodes in the network, using an efficient neighborhood broadcast scheme (akin to Abstract Regions). This information coupled with a utility function representing the intrinsic value of each possible state (such as a choice of parent node) allows nodes to make informed decisions that account for the non-local energy impact. In the paper we show that IDEA can significantly improve energy load distribution and network longevity with low overhead.

The PC Meeting Protocol

I am always surprised at how chaotic program committee meetings tend to be. Although most people have served on several PCs, it seems that a lot of the same procedural questions and issues come up each time, and it would be helpful to establish a common protocol for the community to maintain order. Having just gone through the SIGCOMM TPC meeting (with a whopping 50 PC members - it was like being a delegate at the UN) I started thinking about some of the things we could possibly standardize to make the process run more smoothly. (By the way, Geoff and KK did an awesome job running the meeting - the problems outlined below are common to *all* PCs I have been on!) Michael Mitzenmacher not-live-blogged about this meeting here.

The first is laying down the ground rules. Program chairs tend to assume that PC members know basic things like not to leak information during the PC meeting (emailing your students or colleagues when the paper is being discussed), not to express an opinion on papers you didn't review, and not to compare the paper to the version that was rejected last year. This stuff seems obvious but it's amazing how often people forget.

The order in which papers are discussed is another important decision. In this case there is no one-size-fits-all model. In my opinion, the best model is to group papers by broad topic areas, and within each area discuss the best and worst papers in the group first, followed by those in the middle of the group. This helps to calibrate things and keeps the discussion focused on a set of papers with some commonality. The worst model, I think, is to discuss papers by order of submission number (which is effectively random), since people don't get calibrated that way.

Keeping the discussion to a time limit is very important. Many PC members think that it's their job to hold forth about every minute detail of the paper during the meeting. Once, there was a case where the lead discussant went on for about 6 minutes about a paper, and when finally cut off and asked what he thought the decision should be, said "I don't care!" PC members need to remember that the audience for this discussion is the chairs and the other reviewers (the other PC members are usually reading email or otherwise ignoring discussion of papers they didn't read). So keep it short and sweet, and respect everyone's time.

Dealing with conflicts is always a huge pain. It is disruptive to ask people to leave the room and then call them back in later. It would be awesome if HotCRP could automate this (coming up with a paper discussion schedule to minimize the number of times someone has to leave the room). I'd like an iPhone app that automatically buzzes people when they should leave and come back into the room -- a lot of time is wasted in PC meetings with these context switches.

It has to be recognized that reaching consensus is not always possible. PC members have to accept that they will win some and lose some. I dislike it when the discussion comes down to a vote amongst the reviewers, since that is rarely a good way to decide these things, but at least it is quick and painless.

The last issue is the role of shepherding. This should be explained clearly by the PC chairs at the start of the meeting. Personally, I am in favor of shepherding every paper for a major conference, with the goal being to ensure that the final paper really satisfies the reviewers' main comments and the writing is cleaned up (as it often needs to be). In general, shepherding should not be used to get the authors to run new experiments or change something technical in the design -- it should be limited to changes in wording or positioning of the work. This question comes up at every PC meeting I've been on and setting expectations early would make things easier.

Check out my related post on the psychology of program committees for a behind-the-scenes look at what happens behind the closed doors.