Skip to main content

Posts

Death by peer review

I recently had the occasion to give some advice a friend who was considering making the switch from industry to academia. One of my key pieces of advice was to keep in mind that success (or failure) in academia is largely based on peer review -- by program committees, proposal review panels, tenure committees. While peer review has many good things going for it, it can also be extremely, dishearteningly random. Being an academic means living your life one peer-review decision to the next, and in many cases, those decisions are simply not the right ones. After a while, a string of semi-random decisions can be psychologically draining.

The law of large numbers certainly applies here. Good work eventually gets published and funded, given enough iterations. Good researchers get their papers in, eventually. Peer review feedback can be incredibly helpful for refining a piece of work and improving it over time. But in the vast majority of cases, papers or proposals, whether accepted or reje…
Recent posts

Why I gave your paper a Strong Accept

See also: Why I gave your paper a Strong Reject

I know this blog is mostly about me complainingaboutacademics, but there's a reason I stay engaged with the research community: I learn stuff. Broadly speaking, I think it's incredibly important for industry to both stay abreast of what's going on in the academic world, as well as have some measure of influence on it. For those reasons, I serve on a few program committees a year and do other things like help review proposals for Google's Faculty Research Award program.

Apart from learning new things, there are other reasons to stay engaged. One is that I get a chance to meet and often work with some incredible colleagues, either professors (to collaborate with) or students (to host as interns and, in many cases, hire as full-time employees later on).

I also enjoy serving on program committees more than just going to conferences and reading papers that have already been published. I feel like it's part of my job to gi…

Why I gave your paper a Strong Reject

Also see: Why I gave your paper a Strong Accept.

I'm almost done reviewing papers for another conference, so you know what that means -- time to blog.

I am starting to realize that trying to educate individual authors through my witty and often scathing paper reviews may not be scaling as well as I would like. I wish someone would teach a class on "How to Write a Decent Goddamned Scientific Paper", and assign this post as required reading. But alas, I'll have to make do with those poor souls who stumble across this blog. Maybe I'll start linking this post to my reviews.

All of this has probably been said before (strong reject) and possibly by me (weak accept?), but I thought I'd share some of the top reasons why I tend to shred papers that I'm reviewing.

(Obligatory disclaimer: This post represents my opinion, not that of my employer. Or anyone else for that matter.)

The abstract and intro suck. By the time I'm done reading the first page of the paper…

Everything I did wrong as a professor

I really screwed things up as a young faculty member at Harvard. It worked out OK in the end, but, man, I wish I could go back in time to when I was a new professor and give my younger self some much-needed advice. No, not the "you shouldn't be a professor, get another kind of job" advice -- I wouldn't have listened to that -- but one of the reasons I ended up leaving academia is that I burned myself out. Maybe that could have been avoided had I taken a different approach to the job.

What did I get wrong? Let me count the ways...

Working on too many projects at once. I thrive on having many balls in the air. As a junior faculty member, though, I probably should have stayed focused on just one or two juicy projects, and let all the others fall to the side. I did not have a good filter for thinking about which projects I should take on and where they might lead. It was difficult to say no to any new research direction, since for all I knew it might lead somewhere great…

Academics, we need to talk.

Although I made the move to industry a bit more than five years ago, I still serve on program committees and review articles for journals and the like. So it's painful for me to see some of my academic colleagues totally botch it when it comes to doing industry-relevant research. Profs, grad students: we need to talk.

(Standard disclaimer: This is my personal blog and the opinions expressed here are mine alone, and most certainly not that of my employer.)

Of course, many academics do a great job of visionary, out-of-the-box, push-the-envelope research that inspires and drives work in industry. I'm talking about stuff like Shwetak Patel's increasingly insane ways of inferring activities from random signals in the environment; Dina Katabi's rethinking of wireless protocols; and pretty much anything that David Patterson has ever done. Those folks (and many others) are doing fine. Keep it up.

But the vast majority of papers and proposals I read are, well, crap. Mostly I…

What I learned about mobile usage in Indonesia

A couple of weeks ago I traveled to Jakarta to understand mobile (and especially mobile browser) usage in Indonesia. Indonesia is a huge country with a population of nearly 250 million people and a vast number of them are getting online. For many, smartphones are the first and only device they use for accessing the Internet. I wanted to share some of the things I learned interviewing a number of Indonesian smartphone users.


I want to emphasize that this is my personal blog, and the opinions expressed here are mine, and not that of my employer.

Some of my key takeaways from the week...

Smartphones are central to users' lives
For everyone I interviewed, their smartphone was absolutely central to their life and was a major window to the outside world. For nearly all of these users, the smartphone is the first and only Internet-connected device they own, and they rely on their phones a great deal. Desktop or laptop Internet usage was limited to office workers or students, and even the…

A modest proposal: SOSIGCOMMOBIXDI

I have a problem: there are way too many conferences to attend. Even worse, the degree of overlap between the systems, mobile, and networking communities means that I am basically running into the same people at all of these events. You have a problem, too: You are paying money (and time) to attend all of these separate conferences.

Conservatively, there are five "top tier" conferences that are "must attend" events every year: SOSP/OSDI, NSDI, MobiSys, MobiCom, SIGCOMM. (Not to mention excellent venues like USENIX ATC, EuroSys, CoNext, SenSys, the list goes on.) And then all of the "smaller workshops because we don't like how big the conferences are but you pretty much have to go anyway": HotOS, HotMobile, HotNets.

Realistically, nobody makes it to all of these events (unless you're, say, a poor junior faculty member going for tenure and have to show your place in as many places as possible). So you pick and choose based on whether you have a pap…