Volatile and Decentralized

Making universities obsolete

2012-01-23T22:55:00.000-08:00

Sebastian Thrun recently announced that he was leaving Stanford to found a free, online university called Udacity. This is based on his experiences teaching the famous intro to AI class, for free, to 160,000 students online.

Is this just Education for the Twitter Generation? Or truly a revolution in how we deliver higher education? Will this ultimately render universities obsolete?

I want to ponder the failings of the conventional higher education model for a minute and see where this leads us, and consider whether something like Udacity is really the solution.

Failure #1: Exclusivity.

In Sebatian's brilliant talk at DLD, he talks about being embarrassed that he was only able to teach a few tens of students at a time, and only to those who can afford $30,000 to attend Stanford. I estimate that I taught fewer than 500 students in total during my eight years on the faculty at Harvard. That's a pretty poor track record by any stretch.

It gets worse. I know plenty of faculty who love to give tough courses, in which they would teach really hard material at the beginning of the semester to "weed out" the weaker students, sometimes being left with only 2 or 3 really committed and really good students in the class. This is so much more satisfying as a professor, since you don't need to worry about tutoring the weaker students, and the fewer students you have, the less work you have to do grading and so on. There is no penalty for doing this - and rarely any incentive given for teaching a larger, more popular course.

Exclusivity is necessary when you only have so much classroom space, or so many dorms, or so many dining halls, so you have to be selective about who enters the hallowed gates of the university. It's also a way of maintaining a brand: even schools, like Harvard, with a "distance education" component go to great lengths to differentiate the "true" Harvard education from a "distance learning certificate," lest they raise the ire of the Old Boys' Network by watering down what it means to get a Harvard degree (not unlike the reaction they got when they started admitting women, way way back in 1977).

Failure #2: Grades.

Can someone remind me why we still have grades? I like what Sebastian says (quoting Salman Khan) about learning to ride a bicycle: It's not as if you get a D learning to ride a bike, then you stop and move onto learning the unicycle. Shouldn't the goal of every course be to get every student to the point of making an A+?

Apparently not. The common argument is that we need grades in order to differentiate the "good" from the "bad" students. Presumably the idea is that if you can't get through a course in the 12-to-13 week semester then you deserve to fail, regardless of whatever is going on in your life and whether you could have learned everything over a longer time span, or with more help, or whatever. And the really smart students, the ones who nail it the first time, and make A's in every class, need to float to the top so they get the first dibs on good jobs or law school or medical school or whatever rewards they have been working all of their young lives to achieve. It would not be fair if everyone made an A+ -- how would the privileged and smart kids gain any advantage over the less privileged, less intelligent kids?

It seems to me that this is completely at odds with the idea of education.

Failure #3: Lectures.

As Sebastian says, universities have been using the lecture format for more than a thousand years. I used to tell students that they were required to come to my lectures, and never provided my lectures by video, lest the students skipped class and watched it on YouTube from their dorms instead. Mostly this was to ensure that everybody in the class got the benefit of my dynamic and entertaining lecture style, which I worked so hard to perfect over the years (complete with a choreographed interpretive dance demonstrating the movement of the disk heads during a Log-Structured Filesystem cleaning operation.) But mostly it was to boost my ego and get some gratification for working so hard on the lectures, by having the students physically there in class as an audience.

Implications

I'm not sure whether Udacity and Khan Academy and iTunes University are really the solution to these problems. Clearly they are not a replacement for the conventional university experience -- you can't go to a frat party, or join a Finals Club, or make out in the library stacks while getting your degree from Online U. (At least not yet.)

But I think there are two important things that online universities bring to the table: (1) Broadening access to higher education, and (2) Leveraging technology to explore new approaches to learning.

The real question is whether broadening access ends up reinforcing the educational caste system: if you're not smart or rich enough to go to a "real university," you become one of those poor, second-class students with a certificate Online U. Would employers or graduate schools ever consider such a certificate, where everyone makes an A+, equivalent to an artium baccalaureus from the Ivy League school of your choice?

If not, is that because we truly believe that students are getting a better education sitting in a dusty classroom and having paid the proverbial $30,000 a year rather than doing the work online? This reminds me of my friends who have been through medical school, where the conventional wisdom is that doctors need to be trained using the classical methods (unbelievable amounts of rote memorization, soul-destroying clinical rotations and countless overnight shifts) because that's how it's been done for hundreds of years -- not because anybody thinks it yields better-trained doctors.

And I think universities have a long way to go towards embracing new technologies and new ways of teaching students. Sebastian makes a great point about the online AI class feeling more "intimate" to some students, in part because it really is a feeling of a one-on-one experience watching a video: you're not sitting in a big lecture hall surrounded by a bunch of other students, you're at home, in your PJs, drinking a beer and watching the video on your own laptop. A lot of this also has to do with Sebastian's teaching style, using a series of short quizzes that are auto-graded by the system. It is not just a lecture. For this reason I think that replacing live courses with videotaped lectures is not going far enough (and may in fact be detrimental).

Another benefit of the video delivery model is that you can replay it infinitely many times. Missed a point? Confused? Rewind and watch it again. What about questions? In large courses almost nobody asks questions, apart from the really smart students who should shut the hell up and not ask questions anyway. There are plenty of ways to deal with questions in an online course format, just not live, during a (limited time) lecture in which your question is likely going to annoy the rest of the class who almost certainly gets it already.

Risks

I'm going to close this little rant with a few caveats. It's fashionable to talk about "University 2.0" and How the Internet Changes Everything and disruptive technologies and all that. But a shallow, 18-minute video on the first 200 years of American History can't replace conventional coursework, deep reading, and essays. You can't tweet your way through college. Learning and teaching are hard work, and need to be taken seriously by both the student and educator.

Although expanding access to education is a great thing, it's simply not the case that everyone is smart enough to do well in any subject. For example, I'm terrible at math (which is why I'm a systems person, natch), and damn near failed to complete my CS theory course requirement at Berkeley as a result. Education should give everyone the opportunity to succeed, but the ultimate responsibility (and raw ability) comes down to the student.

Finally, it goes without saying that the most important experiences I ever had in college were outside of the classroom. I'm not just talking about staying up late and watching "Wayne's World" for the millionth time while drinking Zima, I'm talking about doing research, building things, learning from and being inspired by my fellow students. Making lectures obsolete is one thing; but I'm not sure there can ever be an online replacement for The College Experience writ large. Though 4Chan seems to be a pretty close approximation.

Research without walls

2011-11-07T21:36:00.000-08:00

I recently signed the Research Without Walls pledge, which says that I will not do any peer review work for conferences, journals, or other scientific venues that do not make the results available for free via the Web. Like many scientists, I commit hundreds of hours a year to serving on program committees and reviewing journal papers, but the result of that (volunteer) work is essentially that the research results get locked behind a copyright license that is inconsistent with the way in which scientists actually disseminate their results -- for free, via the Web.

I believe that there is absolutely no reason for research results, especially those supported by public funding, not to be made open to the entire world. It's time for the computer science research community to move in this direction. Of course, this is going to mean a big change in the role of the professional societies, such as ACM and IEEE. It's time we made that change, as painful as it might be.

What is open access?

The issue of "open access research" often gets confused with questions such as where the papers are hosted, who owns the copyright, and whether authors are allowed to post their own papers on their website. In most cases, copyright in research publications is not held by the authors, but rather the professional societies that organize a conference or run a journal. For example, ACM and IEEE typically require authors to assign copyright to them, although they might grant the author a license to post their own research papers on their website. However, allowing authors to post papers on the Web is not the same as open access. It is an extremely limited license: posting papers on the Web does not give other scientists or students the right to share or archive those papers, or for anyone to use them for any other purpose other than downloading them for personal use. It is not unlike going to the library and borrowing a book; you still have to return it later, and you can't make copies for others.

With rare exception, every paper I have published is available for download on my website. In most cases, I have a license to do this; in others, I am probably in violation of copyright for doing so. The idea that I might get a cease-and-desist letter one day asking me to take down my own scientific papers bothers me to no end. I worked hard on those papers, and in most cases, spent hundreds of thousands of dollars of public funding to undertake the research that went into each of them.

For most of these publications, I even paid hundreds of dollars to the professional societies -- for membership fees and conference registrations for myself and my students -- to present the work at the associated conference. But yet, I don't own copyright in most of those works, and the main beneficiaries of all of this work are organizations like the ACM. It seems to me that these results should be open for everyone to benefit from, since, well, "we" (meaning, the taxpayers) paid for them.

ACM's Author-izer Service

Recently, the ACM announced a new service called the "Author-izer" (whoever came up with this name will be first against the wall when the revolution comes), that allows authors to generate free links to their publications hosted on the ACM Digital Library. This is not open access, either: this is actually a way for ACM to discourage the spread of "rogue posting" of PDF files and monetize access to the content down the road. For example, those free links will stop working when the website hosting them moves (e.g., when a student graduates). Essentially, ACM wants to control all access to "its" research library, and for good reason: it brings in a lot of revenue.

USENIX's open access policy

USENIX has a much more sane policy. Back in 2008, USENIX announced that all of their conference proceedings would be open access, and indeed you can download PDFs of all USENIX papers from the corresponding conference website (see, for example, http://www.usenix.org/events/hotcloud11/tech/ for the proceedings from HotCloud'11).

USENIX does not ask authors to assign copyright to them. Instead, for one year from the publication date, USENIX gets an exclusive license to publish the work (both in print and electronic form), with the usual license granted back to the author to post copies on their website. After the one-year exclusivity period, USENIX retains a non-exclusive license to distribute the work forever. This is a good policy, though in my opinion it does not go far enough: USENIX does not require authors to release their work under an open access license. USENIX is kind enough to post PDFs for free on the Web, but tomorrow, USENIX could reverse this decision and put all of those papers behind a paywall, or take them down entirely. (No, I don't think this is going to happen, but you never know.)

University open access initiatives

Another way to fight back is for your home institution to require all of your work be made open. Harvard was one of the first major universities to do this. This ambitious effort, spearheaded by my colleague Stuart Shieber, required all Harvard affiliates to submit copies of their published work to the open-access Harvard DASH archive. While in theory this sounds great, there are several problems with this in practice. First, it requires individual scientists to do the legwork of securing the rights and submitting the work to the archive. This is a huge pain and most folks don't bother. Second, it requires that scientists attach a Harvard-supplied "rider" to the copyright license (e.g., from the ACM or IEEE) allowing Harvard to maintain an open-access copy in the DASH repository. Many, many publishers have pushed back on this. Harvard's response was to allow its affiliates to get an (automatic) waiver of the open-access requirement. Well, as soon as word got out that Harvard was granting these waivers, the publishers started refusing to accept the riders wholesale, claiming that the scientist could just request a waiver. So the publishers tend to win.

Creative Commons for research publications

The only way to ensure that research is de jure open access, rather than merely de facto, is by baking the open access requirement into the copyright license for the work. This is very much in the same spirit as the GPL is for software licensing. What I really want is for all research to be published under something like a Creative Commons Attribution 3.0 Unported license, allowing others to share, remix, and make commercial use of the work as long as attribution is given. This kind of license would prevent professional organizations from locking down research results, and give maximum flexibility for others to make use of the research, while retaining the conventional expectations of attribution. The "remix" clause might seem a little problematic, given that peer review expects original results, but the attribution requirement would not allow someone to submit work that is not their own and claim authorship. And there are many ways in which research can be legitimately remixed: incorporated into a talk, class notes, or collection, for example.

What happens to the publishers?

Traditional scientific publishers, like Elsevier, go out of business. I don't have a problem with that. One can make a strong argument that traditional scientific publishers have fairly limited value in today's world. It used to be that scientists needed publishers to disseminate their work; this has not been true for more than a decade.

Professional organizations, like ACM and IEEE, will need to radically change what they do if they want to stay alive. These organizations do many other things other than run conferences and journals. Unfortunately, a substantial amount of their operating budget comes from controlling access to scientific literature. Open access will drastically change that. Personally, I'd rather be a member of a leaner, more focused professional society that can focus its resources on education and policymaking than supporting a gazillion "Special Interest Groups" and journals that nobody reads.

Seems to me that USENIX strikes the right balance: They focus on running conferences. Yes, you pay through the nose to attend these events, though it's not any more expensive than a typical ACM or IEEE conference. I really do not buy the argument that an ACM-sponsored conference, even one like SOSP, is any better than one run by USENIX. Arguably USENIX does a far better job at running conferences, since they specialize in it. ACM shunts most of the load of conference organization onto inexperienced academics, with predictable results.

A final word

I can probably get away with signing the Research Without Walls pledge because I no longer rely on service on program committees to further my career. (Indeed, the pledge makes it easier for me to say no when asked to do these things.) Not surprisingly, most of the signatories of the pledge have been from industry. To tell an untenured professor that they should sign the pledge and, say, turn down a chance to serve on the program committee for SOSP, would be a mistake. But this is not to say that academics can't promote open access in other ways: for example, by always putting PDFs on their website, or preferentially sending work to open access venues.

ObDisclaimer: This is my personal blog. The views expressed here are mine alone and not those of my employer.

Highlights from SenSys 2011

2011-11-04T21:09:00.000-07:00

ACM SenSys 2011 just wrapped up this week in Seattle. This is the premier conference in the area of wireless sensor networks, although lately the conference has embraced a bunch of other technologies, including sensing on smartphones and micro-air vehicles. It's an exciting conference and brings together a bunch of different areas.

Rather than a full trip report, I wanted to quickly write up two highlights of the conference: The keynote by Michel Maharbiz on cybernetic beetles (!), and an awesome talk by James Biagioni on using smartphone data to automatically determine bus routes and schedules.

Keynote by Mich Maharbiz - Cyborg beetles: building interfaces between the synthetic and the multicellular

Mich is a professor at Berkeley and works in the interface between biology and engineering. His latest project is to adding a "remote control" circuit to a live insect -- a large beetle -- allowing one to control the flight of the insect. Basically, they stick electrodes into the beetle's brain and muscles, and a little microcontroller mounted on the back of the insect sends pulses to cause the insect to take off, land, and turn. A low-power radio on the microcontroller lets you control the flight using, literally, a Wii Mote.

Oh yes ... this is real.

There has been a lot of interest in the research community in building insect-scale flying robots -- the Harvard RoboBees project is just one example. Mich's work takes a different approach: let nature do the work of building the flyer, but augment it with remote control capabilities. These beetles are large enough that they can carry a 3 gram payload, can fly for kilometers at a time, and live up to 180 days.

Mich's group found that by sending simple electrical pulses to the brain and muscles that they could activate and deactivate the insect's flying mechanism, causing it to take off and land. Controlling turns is a bit more complicated, but by stimulating certain muscles behind the wings they can cause the beetle to turn left or right on command.

They have also started looking at how to tap into the beetle's sensory organs -- essentially implanting electrodes behind the eye and antennae -- so it is possible to take electrical recordings of the neural activity. And they are also looking at implanting a micro fuel cell that generates electricity from the insect's hemolymph -- essentially turning its own internal fuel source into a battery.

Mich and I were actually good friends while undergrads at Cornell together. Back then he was trying to build a six-legged insect inspired walking robot. I am not sure if it ever worked, but it's kind of amazing to run into him some 15 years later and see he's still working on these totally out-there ideas.

EasyTracker: Automatic Transit Tracking, Mapping, and Arrival Time Prediction Using Smartphones
James Biagioni, Tomas Gerlich, Timothy Merrifield, and Jakob Eriksson (University of Illinois at Chicago)

James, a PhD student at UIC, gave a great talk on this project. (One of the best conference talks I have seen in a long time. I found out later that he won the best talk award - well deserved!) The idea is amazing: To use GPS data collected from buses to automatically determine both the route and the schedule of the bus system, and give users real-time indications of expected arrival times for each route. All the transit agency has to do is install a GPS-enabled cellphone in each bus (and not even label which bus it is, or which route it would be taking - routes change all the time anyway). The data is collected and processed centrally to automatically build the tracking system for that agency.

The system starts with unlabeled GPS traces to extract routes and locations / times of stops. They use kernel density estimation with a Gaussian kernel function to “clean up” the raw traces and come up with clean route information. Some clever statistical analysis to throw out bogus route data.

To do stop extraction, they use a point density estimate with thresholding for each GPS location, which results in clusters at points where buses tend to stop. This will produce a bunch of "fake" stops at traffic lights and stop signs - the authors decided to err on the side of too many stops than too few, so they consider this to be an acceptable tradeoff.

To extract the bus schedule, they look at the arrival times of buses on individual days and use k-means clustering to determine the “centroid time” of each stop. This works fine for first stop on route (which should be close to true schedule). For downstream stops this data ends up being to be too noisy, so instead they compute the mean travel time to each downstream stop.

Another challenge is labeling buses: Need to know which bus is coming down the road towards you. For this, they use a history of GPS traces from each bus, and build an HMM to determine which route the bus is currently serving. Since buses change routes all the time, even during the same day, this has to be tracked over time. Finally, for arrival time prediction, they use the previously-computed arrival time between stops to estimate when the bus is likely to arrive.

I really liked this work and the nice combination of techniques used to take some noisy and complex sensor data and distill it into something useful.

Software is not science

2011-11-02T11:57:00.000-07:00

Very often I see conference paper submissions and PhD thesis proposals that center entirely on a piece of software that someone has built. The abstract often starts out something like this:

We have designed METAFOO, a sensor network simulator that accurately captures hardware level power consumption. METAFOO has a modular design that achieves high flexibility by allowing new component models to be plugged into the simulation. METAFOO also incorporates a Java-based GUI environment for visualizing simulation results, as well as plugins to MATLAB, R, and Gnuplot for analyzing simulation runs....

You get the idea. More often than not, the paper reads like a technical description of the software, with a hairy block diagram with a bunch of boxes and arrows and a detailed narrative on each piece of the system, what language it's implemented in, how many lines of code, etc. The authors of such papers quite earnestly believe that this is going to make a good conference submission.

While this all might be very interesting to someone who plans to use the software or build on it, this is not the point of a scientific publication or a PhD dissertation. All too often, researchers -- especially those in systems -- seem to confuse the scientific question with the software artifact that they build to explore that question. They get hung up on the idea of building a beautiful piece of software, forgetting that the point was to do science.

When I see a paper submission like this, I will start reading it in the hopes that there is some deeper insight or spark of inspiration in the system design. Usually it's not there. The paper gets so wrapped up in describing the artifact that it forgets to establish the scientific contributions that were made in developing the software. These papers do not tend to get into major conferences, and they do not make a good foundation for a PhD dissertation.

In computer systems research, there are two kinds of software that people build. The first class comprises tools used to support other research. This includes things like testbeds, simulators, and so forth. This is often great, and invaluable, software, but not -- in and of itself -- the point of research itself. Countless researchers have used ns2, Emulab, Planetlab, etc. to do their work and without this investment the community can't move forward. But all too often, students seem to think that building a useful tool equates to doing research. It doesn't.

The second, and more important, kind of software is a working prototype to demonstrate an idea. However, the point of the work is the idea that it embodies, not the software itself. Great examples of this include things like Exokernel and Barrelfish. Those systems demonstrated a beautiful set of concepts (operating system extensibility and message-passing in multicore processors respectively), but nobody actually used those pieces of software for anything more than getting graphs for a paper, or maybe a cute demo at a conference.

There are rare exceptions of "research" software that took on a life beyond the prototype phase. TinyOS and Click are two good examples. But this is the exception, not the rule. Generally I would not advise grad students to spend a lot of energy on "marketing" their research prototype. Chances are nobody will use your code anyway, and time you spend turning a prototype into a real system is time better spent pushing the envelope and writing great papers. If your software doesn't happen to embody any radical new ideas, and instead you are spending your time adding a GUI or writing documentation, you're probably spending your time on the wrong thing.

So, how do you write a paper about a piece of software? Three recommendations:

Put the scientific contributions first. Make the paper about the key contributions you are making to the field. Spell them out clearly, on the first page of the paper. Make sure they are really core scientific contributions, not something like "our first contribution is that we built METAFOO." A better example would be, "We demonstrate that by a careful decomposition of cycle-accurate simulation logic from power modeling, we can achieve far greater accuracy while scaling to large numbers of nodes." Your software will be the vehicle you use to prove this point.
Decouple the new ideas from the software itself. Someone should be able to come along and take your great ideas and apply them in another software system or to a completely different problem entirely. The key idea you are promoting should not be linked to whatever hairy code you had to write to show that the idea works in practice. Taking Click as an example, its modular design has been recycled in many, many other software systems (including my own PhD thesis).
Think about who will care about this paper 20 years from now. If your paper is all about some minor feature that you're adding to some codebase, chances are nobody will. Try to bring out what is enduring about your work, and focus the paper on that.

Do we need to reboot the CS publications process?

2011-09-26T20:31:00.000-07:00

My friend and colleague Dan Wallach has an interesting piece in this month's Communications of the ACM on Rebooting the CS Publication Process. This is a topic I've spent a lot of time thinking about (and ranting about) the last few years and thought I should weigh in. The TL;DR for Dan's proposal is something like arXiv for CS -- all papers (published or not) are sent to a centralized CSPub repository, where they can be commented on, cited, and reviewed. Submissions to conferences would simply be tagged as such in the CSPub archive, and "journals" would simply consist of tagged collections of papers.

I really like the idea of leveraging Web 2.0 technology to fix the (broken) publication process for CS papers. It seems insane to me that the CS community relies on 18th-century mechanisms for peer review, that clearly do not scale, prevent good work from being seen by larger audiences, and create more work for program chairs having to deal with deadlines, running a reviewing system, and screening for plagiarized content.

Still, I'm concerned that Dan's proposal does not go far enough. Mostly his proposal addresses the distribution issue -- how papers are submitted and archived. It does not fix the problem of authors submitting incremental work. If anything, it could make the problem worse, since I could just spam CSPub with whatever random crap I was working on and hope that (by dint of my fame and amazing good looks) it would get voted up by the plebeian CSPub readership irrespective of its technical merit. (I call this the Digg syndrome.) In the CSPub model, there is nothing to distinguish, say, a first year PhD student's vote from that of a Turing Award winner, so making wild claims and writing goofy position papers is just as likely to get you attention as doing the hard and less glamorous work of real science.

Nor does Dan's proposal appear to reduce reviewing load for conference program committees. Being a cynic, it would seem that if submitting a paper to SOSP simply consisted of setting a flag on my (existing) CSPub paper entry, then you would see an immediate deluge of submissions to major conferences. Authors would no longer have to jump through hoops to submit their papers through an arcane reviewing system and run the gauntlet of cranky program chairs who love nothing more than rejecting papers due to trivial formatting violations. Imagine having your work judged on technical content, rather than font size! I am not sure our community is ready for this.

Then there is the matter of attaining critical mass. arXiV already hosts the Computing Research Repository, which has many of the features that Dan is calling for in his proposal. The missing piece is actual users. I have never visited the site, and don't know anyone -- at least in the systems community -- who uses it. (Proof: There are a grand total of six papers in the "operating systems" category on CORR.) For better or worse, we poor systems researchers are programmed to get our publications from a small set of conferences. The best way to get CSPub to have wider adoption would be to encourage conferences to use it as their main reviewing and distribution mechanism, but I am dubious that ACM or USENIX would allow such a thing, as it takes a lot of control away from them.

The final question is that of anonymity. This is itself a hotly debated topic, but CSPub would seem to require authors to divulge authorship on submission, making it impossible to do double-blind reviewing. I tend to believe that blind reviewing is a good thing, especially for researchers at less-well-known institutions who can't lean on a big name like MIT or Stanford on the byline.

The fact is that we cling to our publication model because we perceive -- rightly or wrongly -- that there is value in the exclusivity of having a paper accepted by a conference. There is value for authors (being one of 20 papers or so in SOSP in a given year is a big deal, especially for grad students on the job market); value for readers (the papers in such a competitive conference have been hand-picked by the greatest minds in the field for your reading pleasure, saving you the trouble of slogging through all of the other crap that got submitted that year); and value for program committee members (you get to be one of the aforementioned greatest minds on the PC in a given year, and wear a fancy ribbon on your name badge when you are at the conference so everybody knows it).

Yes, it's more work for PC members, but not many people turn down an opportunity to be on the OSDI or SOSP program committee because of the workload, and there are certainly enough good people in the community who are willing to do the job. And nothing is stopping you from posting your preprint to arXiv today. But act fast -- yours could be the seventh systems paper up there!

Programming != Computer Science

2011-09-10T17:30:00.000-07:00

I recently read this very interesting article on ways to "level up" as a software developer. Reading this article brought home something that has been nagging me for a while since joining Google: that there is a huge skill and cultural gap between "developers" and "Computer Scientists." Jason's advice to leveling-up in the aforementioned article is very practical: write code in assembly, write a mobile app, complete the exercises in SICP, that sort of thing. This is good advice, but certainly not all that I would want people on my team spending their time doing in order to be true technical leaders. Whether you can sling JavaScript all day or know the ins and outs of C++ templates often has little bearing on whether you're able to grasp the bigger, more abstract, less well-defined problems and be able to make headway on them.

For that you need a very different set of skills, which is where I start to draw the line between a Computer Scientist and a developer. Personally, I consider myself a Computer Scientist first and a software engineer second. I am probably not the right guy to crank out thousands of lines of Java on a tight deadline, and I'll be damned if I fully grok C++'s inheritance rules. But this isn't what Google hired me to do (I hope!) and I lean heavily on some amazing programmers who do understand these things better than I do.

Note that I am not defining a Computer Scientist as someone with a PhD -- although it helps. Doing a PhD trains you to think critically, to study the literature, make effective use of experimental design, and to identify unsolved problems. By no means do you need a PhD to do these things (and not everyone with a PhD can do them, either).

A few observations on the difference between Computer Scientists and Programmers...

Think Big vs. Get 'er Done

One thing that drove me a little nuts when I first started at Google was how quickly things move, and how often solutions are put into place that are necessary to move ahead, even if they aren't fully general or completely thought through. Coming from an academic background I am used to spending years pounding away at a single problem until you have a single, beautiful, general solution that can stand up to a tremendous amount of scrutiny (mostly in the peer review process). Not so in industry -- we gotta move fast, so often it's necessary to solve a problem well enough to get onto the next thing. Some of my colleagues at Google have no doubt been driven batty by my insistence on getting something "right" when they would rather just (and in fact need to) plow ahead.

Another aspect of this is that programmers are often satisfied with something that solves a concrete, well-defined problem and passes the unit tests. What they sometimes don't ask is "what can my approach not do?" They don't always do a thorough job at measurement and analysis: they test something, it seems to work on a few cases, they're terribly busy, so they go ahead and check it in and get onto the next thing. In academia we can spend months doing performance evaluation just to get some pretty graphs that show that a given technical approach works well in a broad range of cases.

Throwaway prototype vs. robust solution

On the other hand, one thing that Computer Scientists are not often good at is developing production-quality code. I know I am still working at it. The joke is that most academics write code so flimsy that it collapses into a pile of bits as soon as the paper deadline passes. Developing code that is truly robust, scales well, is easy to maintain, well-documented, well-tested, and uses all of the accepted best practices is not something academics are trained to do. I enjoy working with hardcore software engineers at Google who have no problem pointing out the totally obvious mistakes in my own code, or suggesting a cleaner, more elegant approach to some ass-backwards page of code I submitted for review. So there is a lot that Computer Scientists can learn about writing "real" software rather than prototypes.

My team at Google has a good mix of folks from both development and research backgrounds, and I think that's essential to striking the right balance between rapid, efficient software development and pushing the envelope of what is possible.

Measuring the mobile web is hard

2011-08-04T21:33:00.000-07:00

I believe strongly that you can't solve a problem until you can measure it. At Google, I've been charged with making the mobile web fast, so naturally, the first step is measuring mobile web performance across a wide range of devices, browsers, networks, and sites. As it turns out, the state of the art in mobile measurement is a complete mess. Different browsers report completely different timings for the same events. There is very little agreement on what metrics we should be optimizing for. Getting good timing out of a mobile device is harder than it should be, and there are many broken tools out there that report incorrect or even imaginary timings.

The desktop web optimization space is pretty complicated, of course, although there's a lot more experience in desktop than in mobile. It's also a lot easier to instrument a desktop web browser than a mobile phone running on a 3G network. Most mobile platforms are fairly closed and fail to expose basic performance metrics in a way that makes it easy for web developers to get at them. We currently resort to jailbreaking phones and running tcpdump and other debugging tools to uncover what is going on at the network and browser level. Clearly it would be better for everyone if this process were simpler.

When we talk about making the mobile web fast, what we are really trying to optimize for is some fuzzy notion of "information latency" from the device to the user. The concept of information latency will vary tremendously from site to site, and depend on what the user is trying to do. Someone trying to check a sports score or weather report only needs limited information from the page they are trying to visit. Someone making a restaurant reservation or buying an airline ticket will require a confirmation that the action was complete before they are satisfied. In most cases, users are going to care most about the "main content" of a page and not things like ads and auxiliary material.

If I were a UX person, I'd say we run a big user study and measure what human beings do while interacting with mobile web sites, using eye trackers, video recordings, instrumented phones -- the works. Unfortunately those techniques don't scale very well and we need something that can be automated.

It also doesn't help that there are (in my opinion) too many metrics out there, many of which have little to do with what matters to the user.

The HTTP Archive (HAR) format is used by a lot of (mostly desktop) measurement tools and is a fairly common interchange format. Steve Souders' httparchive.org site collects HAR files and has some nice tools for visualizing and aggregating them. The HAR spec defines two timing fields for a web page load: onLoad and onContentLoad. onLoad means the time when the "page is loaded (onLoad event fired)", but this has dubious value for capturing user-perceived latency. If you start digging around and trying to find out exactly what the JavaScript onLoad event actually means, you will be hard-pressed to find a definitive answer. The folklore is that onLoad is fired after all of the resources for a given page have been loaded, except that different browsers report this event at different times during the load and render cycle, and JavaScript and Flash can load additional resources after the onLoad event fires. So it's essentially an arbitrary, browser-specific measure of some point during the web page load cycle.

onContentLoad is defined in the HAR Spec as the time when the "Content of the page loaded ... Depeding [sic] on the browser, onContentLoad property represents DOMContentLoad [sic -- should be DOMContentLoaded] event or document.readyState == interactive." Roughly, this seems to correspond to the time when "just" the DOM for the page has been loaded. Normally you would expect this to happen before onLoad, but apparently in some sites and browsers it can happen after onLoad. So, it's hard to interpret what these two numbers actually mean.

The W3C Navigation Timing API goes a long way towards cleaning up this mess by exposing a bunch of events to JavaScript including redirects, DNS lookups, load times, etc. and these times are fairly well-defined. While this API is supported by WebKit, many mobile browsers platforms do not have it enabled; notably iOS (I hope this will be fixed in in iOS5, we will see). The HAR spec will need to be updated with these timings, and someone should carefully document how effectively different browser platforms implement this API in order for it to be really useful.

The W3C Resource Timing API provides an expanded set of events for capturing individual resource timings on a page, which is essential for deep analysis. However, this API is still in the early design stages and there seems to be a lot of ongoing debate about how much information can and should be exposed through JavaScript, e.g., for privacy reasons.

A couple of other metrics depend less on the browser and more on empirical measures, which I tend to prefer.

Time to first byte generally means time to the first byte of the HTTP payload reception on the browser. For WebPageTest, this includes redirects (so redirects are factored into time to first byte). Probably not that useful by itself, but perhaps in conjunction with other metrics. (And God bless Pat Meenan for carefully documenting the measures that WebPageTest reports -- you'd be surprised how often these things are hard to track down.)

WebPageTest also reports time to first paint, which is the first time anything non-white appears in the browser window. This could be as little as a single pixel or a background image, so it's probably not that useful as a metric.

My current favorite metric is the above-the-fold render time, which reports the time for the first screen ("above the fold") of a website to finish rendering. This requires screenshots and image analysis to measure, but it's browser-independent and user-centric, so I like it. It's harder to measure than you would think, because of animations, reflow events, and so forth; see this nice technical presentation for how it's done. Video capture from mobile devices is pretty hard. Solutions like DeviceAnywhere involve hacking into the phone hardware to bring out the video signal, though my preference is for a high-frame-rate video camera in a calibrated environment (which happens to scale well across multiple devices).

One of my team's goals is to provide a robust set of tools and best practices for measuring mobile websites that we can all agree on. In a future post I'll talk some more about the measurements we are taking at Google and some of the tools we are developing.

Making Seattle my home

2011-07-31T22:29:00.000-07:00

I moved to Seattle about 4 months ago, after having lived in Boston for a little more than seven years. Now that I've settled in a bit I thought now would be a good time to write up some of my thoughts on the city and lifestyle here.

The view from Kerry Park in Queen Anne, which was about a 10-minute walk from my house in Queen Anne - before I moved to Wallingford recently.

Upon leaving Boston, I could have moved pretty much anywhere. Most of the cities with a strong tech industry had good job opportunities for my wife, as well, and of course Google has offices in most major cities in the US. So we had plenty of options. We both went to Berkeley for grad school and absolutely love the Bay Area, but we decided not to move back there for a bunch of reasons. The main one being that I would have been working in Mountain View and my wife would have been in SF, and that would have meant a hell of a commute for either of us. It was also not clear that we would have been able to afford a decent house in the Bay Area in any neighborhoods that we would want to live. Our preference would have been to live in the East Bay, bit that would have made the commute problem even worse. With a two-year old son, I'm not willing to go through an hour commute twice a day -- it's simply not worth it to me.

Seattle has a lot of what we were looking for. We live right in the middle of the city (in Wallingford) and for me it's a 10-minute bike commute (to the Google office in Fremont) along the shore of Lake Union, with views of downtown, the Space Needle, and Mount Rainier. It is a fantastic neighborhood with shops, bars, restaurants, playgrounds, and one of the best elementary schools in Seattle (John Stanford) just a few blocks away.

I realized at one point that I probably know more people in Seattle than any other city -- including Boston -- with the University of Washington, Microsoft, Amazon, and Google all here I had this large pre-fab social network already in place. The tech industry is huge here and there seems to be a very active startup community.

The geography here is absolutely stunning. Anywhere you go in Seattle you are surrounded by water, trees, snow-capped mountains. From our house we have a beautiful view to downtown Seattle and Lake Union, with seaplanes taking off and landing overhead. It is also a dense enough city that we can walk or bike to pretty much everything we would need; of course, a big part of this is because we live in Seattle proper, rather than the Eastlake communities of Kirkland, Bellevue, or Redmond, which tend to be more spread out.

This is totally the view from my house in Wallingford. Yes, I would like for that damn tree to not be in the way, but what can you do?

It is no surprise that Seattle is a far more relaxed and progressive place than Boston. A lot of this is, of course, the West Coast vs. East Coast distinction, and in a lot of ways Seattle exemplifies the West Coast aesthetic, much as Boston does the East. Way way more fixie bikes, tattoos, farmers markets, lesbians, hippies, and hippie lesbians with tattoos riding fixie bikes through farmers markets here in Seattle than anywhere in New England. In a lot of ways it's like San Francisco Lite -- a bit less edgy, more approachable, more gentrified, but still very forward-thinking. I feel very much like I belong here, whereas in Boston I always felt like a bit of an outsider.

So far I'm digging the restaurant and cocktail scene in Seattle, which is more adventurous and less stuffy than what you find in Boston (although Boston has some damn good food). I miss really good Chinese food (which is harder to find than you would expect), and surprisingly Seattle doesn't have a ton of great Mexican food options, although I happen to live about a block from the best taco truck in town. Thai and sushi are excellent here, and there seems to be a lot more casual, foodie-type places all over town which do crazy shit like Korean comfort food and ice cream sandwiches.

What am I not so crazy about? Well, I'm on the fence about the weather. The summer has (mostly) been beautiful - 75 degrees, sunny, no humidity at all. Mixed in have been some cooler rainy days that feel out of place for the season. The first couple of months we were here, in April and May, it was rainy and overcast pretty much every day. I take it this is typical for Seattle. The long term question is whether I will be more or less content with this pattern than Boston, which has a much wider temperature range, a couple of months of unbearably cold and snowy weather each year, and sweltering humid summers. It remains to be seen.

Second, everyone in Seattle appears to be white. This is not true of course, but at least in the neighborhoods where I spend most of my time, there is a lot less racial and cultural diversity than Boston. My understanding is that this is due to largely historical reasons where minorities were shut out of many neighborhoods, but the effects persist today. I will ponder this more deeply the next time I'm sitting at a sidewalk café with my dog while sipping an organic soy latte and checking Google+ on my MacBook Pro. It's the thing to do here, you know.

How do you evaluate your grad students?

2011-07-15T07:59:00.000-07:00

One of the issues that I always struggled with as an academic -- and I know many other faculty struggle with -- is keeping grad students on track and giving them useful feedback to help them along in their careers. PhD students often get lost in the weeds at some point (or many points!) during grad school. Of course, part of doing a PhD is figuring out what you want to work on and doing things that might seem to be "unproductive" to the untrained eye. On the other hand, many PhD students grind to a halt, spending months or even years on side projects or simply doing nothing at all. One problem my own students often had was working super hard to submit a paper and then doing almost no new work for 2-3 months while waiting to get the reviews back.

When a student gets stuck in a rut, how do you help them out of it? How do you help students clear a path to productivity?

One thing that many PhD programs lack is any regular and formal evaluation of a student's progress. Harvard never did anything formal, although I tried to get something going it did not last beyond one year -- not enough faculty cared to participate, and we couldn't agree on the process or desired outcomes. At Berkeley, every PhD student got a formal letter every year with a "letter grade" indicating how well you were doing and with some optional comments from your advisor on your overall progress. Although that feedback could have been delivered informally, there was a psychological impact to the formal letter and the idea that all of the professors were meeting in a smoky room to talk about your standing. CMU has its infamous "Black Friday" where all the profs literally do get together to score the grad students. Not having been to CMU, I wonder how this was viewed by the students -- did they find this feedback valuable, stressful, or just plain annoying?

Although this kind of feedback can be useful, for many students it goes in one ear and out the other. I think that part of the reason is that there is often no penalty for doing poorly on a review -- about the only thing a PhD program can threaten you with is kicking you out, and most programs that I know of avoid that unless there's a case where a student has been totally unproductive for a period of several years. It's hard to get kicked out of grad school. By the same token there's little incentive to do well on a review: you're not going to graduate any sooner or get paid more. (Sidebar - should PhD programs pay high-performing grad students bonuses?)

The other issue is that these mechanisms are somewhat open loop in the sense that the student is not expected to lay out a plan and stick to it. Most PhD programs expect students to file some kind of formal plan of study or research leading towards their degree, but it is usually a matter of paperwork and is done just once, or maybe twice, during the course of the program. This has almost no value to the student and is just a matter of paperwork. My feeling is that students would benefit tremendously from a more frequent and formal planning process.

At Google, the approach we use for planning is based on OKRs, or "objectives and key results." Every employee and team is expected to come up with their OKRs for the coming quarter, and score the OKRs from the previous quarter in terms of how much progress was made towards each goal. This is extremely useful process since it gets you thinking about what you need to do over the next 3 months (which seems to be about the right planning horizon for most activities) and you have the chance to reflect on your successes and failures of the previous quarter. It's not expected that you achieve 100% of your goals -- if you are doing so, then your OKRs were not ambitious enough -- you should be shooting for a grade of 70-80%.

I wonder if grad students wouldn't benefit from using something like OKRs for planning their research. A student should be able to say what they are doing over the next 3 months. Looking back on the previous 3 months and grading your progress tells you whether you are generally on track. Having quarterly OKR scores can also help advisors point out where the student needs to improve and documents clear-cut cases where a student has been unproductive (something that both students and advisors are often in denial about). Thoughts?

My experience with Amazon Cloud Player

2011-07-11T21:57:00.000-07:00

As I've posted here before, I'm an avid music fan and collector. A few years ago I decided to go all-digital with my music collection, and since then have mostly refused to buy CDs in favor of digital music online -- mostly from Amazon's (excellent) MP3 store, as well as iTunes. However, this created a new problem: where to keep the music, and how to keep it synced between my various devices -- home laptop, work laptop, work desktop, home desktop, phone, iPad. Lots of people have this problem. My music collection is now more than 50 GB and it's no small feat to keep it synchronized between devices.

For a while I had this crazy scheme where I would only buy new music on my home laptop (the "master" library) which I could sync directly to my phone. From the home laptop I would push new music (using rsync) to my home desktop, which would allow me to listen to it on the stereo at home (via a first-generation Squeezebox player). I would also push it to my desktop at Harvard, so I could listen at work. Once I moved to Google, syncing from home to work became more difficult, although not impossible -- but I could only do so when on the Google VPN, which I can only access on my work laptop. So I modified the aforementioned crazy scheme by syncing from the home laptop to my old desktop machine at Harvard, from which I would pull down new music to my work desktop and laptop. In this way, at least I always knew where the master copy was, and the flow of new data along any edge was always unidirectional, thereby avoiding sync conflicts.

Over time the complexity of this scheme became a real annoyance, especially since I could only buy new music when I was using my home laptop, and syncing to my iPhone and iPad required manually plugging them into that same machine.

So, about a month ago I switched over to using Amazon Cloud Drive, in the hopes that it would fix this problem once and for all. In terms of solving the sync problem, it has been a great success: all of my music now simply lives on Amazon’s servers, and (except for my phone) I don’t need to worry about syncing it to any other devices. For that, Amazon has a Cloud Player app for Android which can pull the music down to my phone directly, without having to stage it on any of my other machines. So I can buy music on the phone, which is immediately available in my Cloud Drive, and I can pull it down to the phone if I want to. (I even bought and downloaded an album to my phone while on an Alaskan Airlines flight, using the in-flight WiFi.) There is an amazing instant-gratification factor here: read about an album on Pitchfork, it’s in your library and on all of your devices in under a minute.

Now, I mostly listen to music at work using the Amazon Cloud Player in my web browser. I don’t even bother syncing a copy to my various machines, though from time to time I do download new music to my home laptop just to have a local backup (in case Amazon goes out of business or something).

Unfortunately, as a user experience goes, I think the web-based Cloud Player has a long way to go:

The web interface is terribly slow, especially with a large music collection such as my own. I have more than 12,000 songs and 1,000 albums in the collection, and opening up the Web-based Cloud Player takes a good 20 seconds. The worst part is scrolling through the music’s “album” view: for some damn reason I always get a yellow spinner when paging through the set of albums, and it can take 20-30 seconds to load the album art for each page so I can see what I want to listen to. Plenty of other websites have solved the problem of previewing large numbers of images, so I can’t understand why Amazon’s site is so slow.
I rarely have problems with the music playback, though keep in mind I am usually listening from Google’s very well provisioned network. (I also happen to live in Seattle, spitting distance from Amazon HQ, though I’m not sure how much that helps.) However, given that I generally have a few dozen other tabs open on my browser in several windows, I have noticed that the Cloud Player running in the background was inducing additional lag. Now, I only run Cloud Player from a different browser (Safari) dedicated to that purpose.
Every 24 hours, the login credentials on the Cloud Player time out and it will stop playback immediately and force me to log in again. This is highly annoying, especially when I’m in the middle of rocking out to Gang Gang Dance. And of course, when I login again, the Cloud Player has forgotten its state so I have to wait 30 seconds to reload my music library and figue out which song I was listening to and fire it up again -- a good minute of rocking-out time lost. I can use Google docs, Gmail, and a bunch of other online services without having to enter my login credentials every day; why can’t Amazon solve this problem? Maybe it’s a licensing issue, but it makes for a painful user experience.
When I first signed up, I had to run Amazon’s MP3 uploader to load my music library into their servers. Given that 99% of the music is available on Amazon’s own MP3 store -- and I suspect a good 15% of my library was originally purchased from same said store -- I was surprised that I had to go through this painful step. To add insult to injury, the uploader for some reason capped bandwidth at 500 Kbps or so, meaning it took nearly a week and a half to upload all my music. (Made even worse because the login credentials would time out every day and would not resume uploading until I logged in again.) I should have been able to upload my entire music library to Amazon in less than 20 hours on my 5 Mbps connection from home, so the artificial cap seems ridiculous.
Somewhat humorously, when I buy new music from Amazon’s MP3 store, there is a delay before it shows up in my library. I can hit reload and watch each song being loaded into the library, and it seems to take a few seconds for each one to appear. I assumed that adding music from Amazon’s MP3 store to my Cloud Drive would be a matter of creating the S3 equivalent of a symlink, so what the hell is going on here?

Just for the record I have not yet tried Google Music, which is in beta, since I am too lazy to move my music collection over yet again. From what I have seen, the web interface is pretty similar so there's not a good enough reason -- yet -- to switch. I'm curious to see what Apple's iCloud is like, but I really like having my music collection liberated from Apple's DRM.

So, I am somewhat surprised at how bare-bones the Amazon Cloud Player is. Given that this is a web app, you would think there would be all kinds of great features that go beyond what you can do inside of a “closed” app like iTunes. For example, there’s no way to share a link to a song or album in Amazon’s MP3 store with Facebook or Twitter -- I have to go search for the listing on Amazon’s MP3 store by hand, and post that manually. This seems like a lost revenue opportunity for Amazon, since it’s hard to share with my friends online that I’m loving the new Bon Iver album and give them a link to buy it (hey, maybe with a referral discount).

Overall it's awesome to use this service and I love having my music everywhere all at once, and not having to manually maintain a library. If the web player were more sophisticated and responsive it would be a slam dunk.

Being Googley

2011-06-23T21:23:00.000-07:00

I've been at Google for almost a year now and have been thinking back on what my expectations of the job would be like compared to what it has turned out to be. This got me thinking about corporate culture in general and how important it is for fostering innovation and being successful.

Google is well known for having a creative work environment with tons of perks -- free food, yoga classes, massage, on-site doctor. Here in Seattle, we can borrow kayaks to take out onto the shipping canal next to the building. (I am fond of telling people this but know full well that I am unlikely to ever take advantage of it.) On the surface these things might seem frivolous, but I think they go a long way towards creating an environment where people are passionate about what they do. The term we use is "being Googley," meaning, doing whatever it is that Google people do: thinking big, focusing on the user, not being evil, etc. On a more day-to-day basis, being Googley means turning out the lights when you leave a conference room, being friendly and helpful to new engineers, being a good citizen.

Google is by no means the only company like this: Yahoo, Facebook, and Amazon are great examples, and many other Internet-era startups follow a similar model. But this is miles away from what I thought corporate life would be like before I joined Google. To be sure, most of my prior exposure to corporate life (that is, before doing my PhD and becoming a professor) was through internships I did at a couple of older technology companies. I spent time writing code at a company that built semiconductor testers, as well as a large electronics company in Japan. I had also visited several large industrial research labs in the US, places that in some cases have been around for more than 50 years. I had a very hard time imagining myself taking a job at any of these companies: the sea of cubicles, drab beige walls, terrible cafeteria food, very few people under 40. Like Dilbert in real life. I wonder how those companies continue to attract top talent when there are places that are so much more appealing to work.

The Google culture is not just about lava lamps in the conference rooms though. The thing that surprised me the most is that there is very little hierarchy in the company: every engineer has the opportunity to create and lead new projects. It's not uncommon for something cool to start up with a few engineers working in their "20% time" -- GMail being one famous example. It is rare for a technical edict to be handed down from on high: projects are usually bottom up and are driven by what the engineers want to accomplish. To be sure, there are projects that could benefit from more adult supervision: things can go off the rails when you don't have enough management. But it's amazing what a merry band of hackers can put together without a lot of imposed structure or artificial constraints from the Pointy Haired Boss. I think the result is that engineers feel a lot of ownership for what they create, rather than feeling like they are just building something to make management happy.

When I was in the Google office in Cambridge, I worked on the team that builds Google's content delivery network -- a huge system that carries a significant chunk of the Internet's traffic (mostly YouTube). Almost everyone on the team was a good 10 years younger than I am (and way smarter, too). There was very little oversight and everyone kind of pitched in to keep the thing running, without anyone having to be told explicitly what to do. I was amazed that you could run such a large, complex project like this, but it seems to work. It's a hacker culture at a large scale. Companies like Facebook and Amazon are run pretty much the same way. All of this seems to turn the conventional wisdom about what it takes to run a successful company upside down. This won't be a surprise to anyone who spent time at startups in the last 10 years, but I'm new to this stuff and surprised that it even works.

The changing face of Computer Science education

2011-06-11T20:31:00.000-07:00

The New York Times has a great article today on changing attitudes towards CS, driven in part by movies like "The Social Network." Apart from the movie's (glossy and inaccurate) depiction of what it's like to be a hacker, there's something else going on here: the fact that CS students can jump in and apply their knowledge to build great things. At Harvard, countless undergrads taking the introductory CS50 class are producing games, websites, and iPhone apps -- some of which are good enough to turn into commercial products. I don't know of any other field where this is possible after taking just a single semester's worth of courses.

Of course, it wasn't always this way. For a very long time, Computer Science education at most Universities was grounded in the abstract and theoretical. Undergraduates rarely got the opportunity to build "real" applications or products. After all, before the advent of cheap, powerful PCs, a department might have one computer for the entire class, and its main purpose was to sit in a walled-off machine room and spit out numbers on a printout -- hardly inspiring. I did my undergrad degree at Cornell, and the first class I took was taught in Scheme -- a language I have never used since -- although the projects were fun for someone like me (implementing a public key cryptosystem, and doing some neat machine vision algorithms). Of course, this was before the Web, iPhones, and Facebook, so CS class "projects" tended to be somewhat dry back then.

Unfortunately, there are still too many vestiges of this old fashioned approach to Computer Science evident in the curriculum. It is largely a generational thing. At Harvard, I had a hell of a time convincing some of the senior faculty that we should be teaching all CS students the fundamentals of computer systems, like how a process works, what a cache is, how to program using threads. (Of course, like most CS degree programs, Harvard still requires all students to learn the finer points of nondeterministic finite state automata and arcane discrete mathematics. Harry Lewis, who teaches this class, once described this to me as "eating your vegetables.")

A few years ago, I was asked to take over teaching CS50, Harvard's introductory CS course. Since the semester was right around the corner, I didn't have time to revamp the course, and agreed to do it only if I could teach the original course material with few changes. I took a look at the existing syllabus. The first lecture was about the "history of computing" and was full of black and white pictures of Babbage and ENIACS and men in horn-rimmed glasses looking over printouts in a terminal room somewhere. This was not a promising start. The next six lectures explained in painful detail -- down to machine instructions and the bit representation of integers -- how to write a single program: How to convert Fahrenheit to Celsius. This being the only program that students saw for the first month or so of the course, it's no wonder that the course did not have broad appeal. This kind of material probably worked very well in the early 1990's, but not so today -- the field has changed, and what students are looking for has changed too.

I passed on teaching CS50, and it's a good thing I did -- Harvard hired David Malan, who is infinitely more energetic and likable than I am, to teach it instead. David completely reworked the course from the perspective of someone who learned CS in the PC and Internet era. He had students hacking iPhone apps, writing PHP and JavaScript, building websites. Over the next few years, enrollment in the course has nearly quadrupled -- it's become one of the "must take" courses at Harvard. He has done an amazing job.

Of course, there is a risk in going too far down this fun, project-oriented route. Computer Science is not a vocational program, and it's important for students to graduate with a deep understanding of the field. It's true that you can do amazing things with existing languages and tools without learning much about the deeper theory and foundations. Still, I think it's great to attract students with a fun, application-oriented course that gets them excited about the field, and hit them later with the more abstract ideas that might seem less relevant at the outset.

One problem is that the classes that follow CS50 are nowhere near as exciting -- they don't have monthly pizza parties and free Facebook schwag at the end of the semester -- so keeping students in the program beyond the intro course can be a challenge. But I think it's important for universities to consider where CS undergrads are coming from and try to meet them there, rather than to teach the way it was done 30 years ago, on a PDP-11 running LISP.

Reflections on Fast, User-Level Networking

2011-05-30T15:05:00.000-07:00

A couple of weeks ago at HotOS, one of the most controversial papers (from Stanford) was entitled "It's Time for Low Latency." The basic premise of the paper is that clusters are stuck using expensive, high-latency network interfaces (generally TCP/IP over some flavor of Ethernet), but it should now be possible to achieve sub-10-microsecond round-trip-times for RPCs. Of course, a tremendous amount of research looked at low-latency, high-bandwidth cluster networking in the mid-1990's, including Active Messages, the Virtual Interface Architecture, and U-Net (which I was involved with as an undergrad at Cornell). A bunch of commercial products were available in this space, including Myrinet (still the best, IMHO) and InfiniBand.

Not much of this work has really taken off in commercial datacenters. John Ousterhout and Steve Rumble argue that this is because the commercial need for low latency networking hasn't been there until now. Indeed, when we were working on this in the 90's, the applications we envisioned were primarily numerical and scientific computing: big matrix multiplies, that kind of thing.

When Inktomi and Google started demonstrating Web search as the "killer app" for clusters, they managed to get away with relatively high-latency, but cheap, Ethernet-based solutions. For these applications, the cluster interconnect was not the bottleneck. Rumble's paper argues that emerging cloud applications are motivating the need for fast intermachine RPC. I'm not entirely convinced of this, but John and Steve and I had a few good conversations about this at HotOS and I've been reflecting on the lessons learned from the "fast interconnect" heyday of the 90's...

Microbenchmarks are evil: There is a risk in focusing on microbenchmarks when working on cluster networking. The standard "ping-pong" latency measurement and bulk transfer throughput measurements rarely reflect the kind of traffic patterns seen in real workloads. Getting something to work on two unloaded machines connected back-to-back says little about whether it will work at a large scale with a complex traffic mix and unexpected load. You often find that real world performance comes nowhere near the ideal two-machine case. For that matter, even "macrobenchmarks" like the infamous NOW-Sort work be misleading, especially when measurements are taken under ideal conditions. Obtaining robust performance under uncertain conditions seems a lot more important than optimizing for the best case that you will never see in real life.

Usability matters: I'm convinced that one of the reasons that U-Net, Active Messages, and VIA failed to take off is that they were notoriously hard to program to. Some systems, like Fast Sockets, layer a conventional sockets API on top, but often suffered large performance losses as a result, in part because the interface couldn't be tailored for specific traffic patterns. And even "sockets-like" layers often did not work exactly like sockets, being different enough that you couldn't just recompile your application to use them. A common example is not being entirely threadsafe, or not working with mechanisms such as select() and poll(). When you are running a large software stack that depends on sockets, it is not easy to rip out the guts with something that is not fully backwards compatible.

Commodity beats fast: If history has proven anything, there's only so much that systems designers are willing to pay -- in terms of complexity or cost -- for performance. The vast majority of real-world systems are based on some flavor of the UNIX process model, BSD filesystem, relational database, and TCP/IP over Ethernet. These technologies are all commodity and can be found in many (mostly compatible) variants, both commercial and open source; few companies are willing to invest time and money to tailor their design for some funky single-vendor user-level networking solution that might disappear one day.

Conference report: HotOS 2011 in Napa

2011-05-11T09:18:00.002-07:00

This week, I served as program chair for the Thirteenth Workshop on Hot Topics in Operating Systems, or HotOS 2011, which took place at the Westin Verasa in Napa, California. HotOS is a unique workshop and one of my favorite venues -- it is the place for systems researchers to put forth their most forward-thinking ideas. Unlike most conferences, HotOS takes 5-page position papers, and it's expected that the submission really represents a position, not a mature piece of technical work condensed into the shorter format.

When it's done right, HotOS is full of great, big sky papers and lots of heated discussions that give the community a chance to think about what's next. In some years, HotOS has been more like an "SOSP preview," with 5-page versions of papers that are likely to appear in a major conference a few months after the workshop. We tried to avoid that this year, and for the most part I think we were successful -- very few papers in this year's HotOS were mature enough to have been considered for SOSP (although that remains to be seen).

I've already blogged about the highly contentious cloud computing panel at HotOS. Here's the rest of the trip report.

Timothy Roscoe holding court at HotOS.

This year I tried to tinker with the conventional conference format in which speakers give 25 minute talks with 5 minutes of questions afterwards. For HotOS, this seems excessive, especially since the papers are so short. Instead, we limited speakers to 10 minutes. There was some pushback on this, but overall I think it was extremely successful: I didn't feel that anyone was rushed, speakers did a great job of staying within the time limits, and by the time a talk started to get boring, it was over.

The other side is we wanted to have room for longer discussions and debates, which often can't happen in the 5 minutes between talks. Too often you hear "let's take that offline," which is code language for "I don't want to get into that in front of the audience." This is a cop-out. At HotOS, after every couple of paper sessions we had a 30-to-45 minute "open mic" session where anybody could ask questions or just rant and rave, which gave plenty of time for more in-depth discussions and debate. At first I was worried that we wouldn't be able to fill up the time, but remarkably there was often plenty of people lined up to take the mic, and lots of great back-and-forth.

A few highlights from this years' HotOS... all of the papers are available online, although they might be limited to attendees only for a while.

Jeff Mogul from HP kicked off the workshop with a talk about reconnecting OS and architecture research. He argued that the systems community is in a rut by demanding that new systems run on commodity hardware, and the architecture community is in a rut by essentially pushing the OS out of the way. He made some great points about the opportunity for OS designs to leverage new hardware features and for the systems community not to be afraid to do so.

To prove this point, Katelin Bailey from UW gave a great talk about how OS designs could leverage fast, cheap NVRAM. The basic idea is to get rid of the gap between memory and disk-based storage altogether, which opens up a wide range of new research directions, like processes which never "die." I find this work very exciting and look forward to following their progress.

Mike Walfish from UT Austin gave a very entertaining talk about "Repair from a Chair." The idea is to allow PC users to have their machines repaired by remote techs by pushing the full software image of their machine into the cloud, where the tech could fix it in a way that the end user can still verify exactly what changes were made to their system. The talk included a nice case study drawn from interviews with Geek Squad and Genius Bar techs -- really cool. My only beef with this idea is that the problem is largely moot when you run applications in the cloud and simply repair the service, rather than the end-user's machine.

Dave Ackley from UNM gave the wackiest, most out-there talk of the conference on "Pursue Robust Indefinite Scalability." I am still not sure exactly what it is about, but the idea seems to be to build modular computers based on a cellular automaton model that can be connected together at arbitrary scales. This is why we have workshops like HotOS -- it would be really hard to get this kind of work into more conventional systems venues. Best quote from the paper: "pledge allegiance to the light cone."

Steve Rumble from Stanford talked about "It's Time for Low Latency," arguing that the time has come to build RPC systems that can achieve 10 microsecond RTTs. Back in the mid-1990s, myself and a bunch of other people spent a lot of time working on this problem, and we called 10 usec the "Culler Constant," since that was the (seemingly unattainable) goal that David Culler set forth for messaging in the Berkeley NOW cluster project. Steve's argument was that the application pull for this -- cloud computing -- is finally here so maybe it's time to revisit this problem in light of modern architectures. I would love to see someone dust off the old work on U-Net and Active Messages and see what kind of performance we can achieve today, and whether there is a role for this kind of approach in modern cluster designs.

Geoff Challen from Univ. Buffalo and Mark Hempstead from Drexel gave the most entertaining talk of the workshop on "The Case for Power-Agile Computing." The idea of the talk was that mobile devices should incorporate multiple hardware components with different power/performance characteristics to support a wide range of applications. As you can see below, Geoff was dressed as a genie and had to say "shazam" a lot.

This might be the first open-shirted presentation ever at HotOS. Let us hope it was the last.

Moises Goldszmidt from MSR gave a really energetic talk on the need for better approaches for modeling and predicting the performance of complex systems. He proposed to use intervention at various points within the system to explore its state space and uncover dependencies. To me, this sounds a lot like the classic system identification problem from control theory, and I would love to see this kind of rigorous engineering approach applied to computer systems performance management.

The traditional Wild and Crazy Ideas session did not disappoint. Margo Seltzer argued that all of the studies assuming users keep cell phones in their pocket (or somewhere on their person) failed to account for the fact that most women keep them in a bag or elsewhere. Good point: I have lost count of how many papers assume that people carry their phones on them at all times. Sam King from UIUC talked about building an app store for household robots, in which the killer app really is a killer app. Dave Andersen from CMU made some kind of extended analogy between systems researchers and an airliner getting ready to crash into a brick wall. (It made more sense with wine.)

We gave away four amazing prizes: Google ChromeOS Laptops! Dave Ackley won the "most outrageous opinion" prize for his wild-eyed thoughts on computer architecture. Vijay Vasudevan from CMU won the best poster award for a poster entitled "Why a Vector Operating System is a Terrible Idea", directly contradicting his own paper in the workshop. Chris Rossbach from MSR and Mike Walfish from UT Austin won the two best talk awards for excellent delivery and great technical content.

Finally, I'd like to thank the program committee and all of the folks at USENIX for helping to make this a great workshop.

How can academics do research on cloud computing?

2011-05-10T08:47:00.000-07:00

This week I'm in Napa for HotOS 2011 -- the premier workshop on operating systems. HotOS is in its 24th year -- it started as the Workshop on Workstation Operating Systems in 1987. More on HotOS in a forthcoming blog post, but for now I wanted to comment on a very lively ~~argument~~ discussion that took place during the panel session yesterday.

The panel consisted of Mendel Rosenblum from Stanford (and VMWare, of course); Rebecca Isaacs from Microsoft Research; John Wilkes from Google; and Ion Stoica from Berkeley. The charge to the panel was to discuss the gap between academic research in cloud computing and the realities faced by industry. This came about in part because a bunch of cloud papers were submitted to HotOS from academic research groups. In some cases, the PC felt that the papers were trying to solve the wrong problems, or making incorrect assumptions about the state of cloud computing in the real world. We thought it would be interesting to hear from both academic and industry representatives about whether and how academic researchers can hope to do work on the cloud, given that there's no way for a university to build something at the scale and complexity of a real-world cloud platform. The concern is that academics will be relegated to working on little problems at the periphery, or come up with toy solutions.

The big challenge, as I see it, is how to enable academics to do interesting and relevant work on the cloud when it's nearly impossible to build up the infrastructure in a university setting. John Wilkes made the point that that he never wanted to see another paper submission showing a 10% performance improvement in Hadoop, and he's right -- this is not the right problem for academics to be working on. Not because 10% improvement is not useful, or that Hadoop is a bad platform, but because those kinds of problems are already being solved by industry. In my opinion, the best role for academia is to open up new areas and look well beyond where industry is working. But this is often at odds with the desire for academics to work on "industry relevant" problems, as well as to get funding from industry. Too often I think academics fall into the trap of working on things that might as well be done at a company.

Much of the debate at HotOS centered around the industry vs. academic divide and a fair bit of it was targeted at my previous blog posts on this topic. Timothy Roscoe argued that academia's role was to shed light on complex problems and gain understanding, not just to engineer solutions. I agree with this. Sometimes at Google, I feel that we are in such a rush to implement that we don't take the time to understand the problems deeply enough: build something that works and move onto the next problem. Of course, you have to move fast in industry. The pace is very different than academia, where a PhD student needs to spend multiple years focused on a single problem to get a dissertation written about it.

We're not there yet, but there are some efforts to open up cloud infrastructure to academic research. OpenCirrus is a testbed supported by HP, Intel, and Yahoo! with more than 10,000 cores that academics can use for systems research. Microsoft has opened up its Azure cloud platform for academic research. Only one person at HotOS raised their hand when asked if anyone was using this -- this is really unfortunate. (My theory is that academics have an allergic reaction to programming in C# and Visual Studio, which is too bad, since this is a really great platform if you can get over the toolchain.) Google is offering a billion core hours through its Exacycle program, and Amazon has a research grant program as well.

Providing infrastructure is only one part of the solution. Knowing what problems to work on is the other. Many people at HotOS bemoaned the fact that companies like Google are so secretive about what they're doing, and it's hard to learn what the "real" challenges are from the outside. My answer to this is to spend time at Google as a visiting scientist, and send your students to do internships. Even though it might not result in a publication, I can guarantee you will learn a tremendous amount about what the hard problems are in cloud computing and where the great opportunities are for academic work. (Hell, my mind was blown after my first couple of days at Google. It's like taking the red pill.)

A few things that jump to mind as ripe areas for academic research on the cloud:

Understanding and predicting performance at scale, with uncertain workloads and frequent node failures.
Managing workloads across multiple datacenters with widely varying capacity, occasional outages, and constrained inter-datacenter network links.
Building failure recovery mechanisms that are robust to massive correlated outages. (This is what brought down Amazon's EC2 a few weeks ago.)
Debugging large-scale cloud applications: tools to collect, visualize, and inspect the state of jobs running across many thousands of cores.
Managing dependencies in a large codebase that relies upon a wide range of distributed services like Chubby and GFS.
Handling both large-scale upgrades to computing capacity as well as large-scale outages seamlessly, without having to completely shut down your service and everything it depends on.

What I'm working on at Google: Making the mobile web fast

2011-05-03T07:54:00.000-07:00

A bunch of people have asked me what I work on at Google these days. When I joined Google last July in the Cambridge office, I worked with the team that runs Google’s content delivery network, which is responsible for caching a vast amount of (mostly video) content at many sites around the world. It is a fantastic project with some great people. My own work focused on building tools to measure and evaluate wide-area network performance and detect performance problems. This was a great “starter project,” and I got to build and deploy some pretty large systems that now run on Google’s worldwide fleet.

Now that I’m in Seattle, I am heading my own team with the charter to make the mobile web fast. By “mobile web”, I mean the entire web as accessed from all mobile devices, not just Google services and not just Android. While Android is a big focus of our work, we care a lot about improving performance for all mobile devices. This project is an outgrowth of Google’s broader make the web faster initiative, which has (to date) largely been focused on desktop web. The parent project has been involved with the Chrome browser, the SPDY protocol (a replacement for HTTP), the WebP image format, numerous network stack enhancements, and more. I see mobile as the next big frontier for the web and there are some huge opportunities to make impact in this space.

This is a hugely exciting project since it touches on many different platforms and cuts across layers. Everyone knows that mobile web usage is taking off: the growth of the mobile web is much, much faster than the growth of the desktop web was back during the dot-com boom. At the same time, the mobile web is painfully slow for most sites and most users. And of course, not everyone has the benefit of using a bleeding-edge 4G phone with LTE network speeds of 25 Mbps, like I do (my current phone is an HTC Thunderbolt, and it’s awesome). For the vast majority of Internet users, a mobile phone will be the only device they ever interact with.

So what are we working on? At a high level we are planning to tackle problems in three broad areas: The mobile devices themselves; the services they connect to; and the networks that connect them. On the device side, we are looking at a wide range of OS, network stack, and browser enhancements to improve performance. On the service side, we are looking at better ways to architect websites for mobile clients, providing tools to help web developers maximize their performance, and automatic optimizations that can be performed by a site or in a proxy service. Finally, at the network layer we are looking at the (sometimes painful) interactions between different layers of the protocol stack and identifying ways to streamline them. There is a huge amount of work to do and I am really fortunate to work with some amazing people, like Steve Souders and Arvind Jain, on this effort.

Our goal is to share solutions with the broader web development community -- we hope to release most of our code as open source, as many other Google projects have done. I also plan to maintain strong ties with the academic research community, since I feel that there is a great opportunity to open up new avenues for mobile systems research, as well as leverage the great work that universities are doing in this space.

I think Google is in a unique position to solve this problem. And, of course, we are hiring! Drop me a line if you are interested in joining our team.

How to write a grant proposal for industry

2011-04-11T22:01:00.000-07:00

I've recently had the pleasure of reviewing proposals for Google's Research Awards program. This is a huge program that gives away millions of dollars a year to a large number of university research projects ranging from machine vision to human-computer interaction to mobile systems. After spending eight years in academia struggling to get funding for my own research, it is quite nice to be on the other side of the table and be the one helping to give away the money, rather than begging for it.

First of all, this is nothing like reviewing proposals for, say, the NSF, where you have 15-20 page proposals and project sizes ranging from 3-5 years and anywhere from one to ten PIs. The Google proposals are (thankfully) short -- only 3 pages -- and generally ask for funding for a couple of grad students for a year, plus some funding for a summer month of a PI and maybe some equipment. So the individual grants are small, which in some sense is frustrating since it's hard to propose anything big and groundbreaking when it's just for a year. Essentially this means that most PIs are only asking for money for things they are already working on, rather than spearheading work in a new direction. (Arguably Google should be giving away more money to fewer schools for longer periods of time, but this is my personal opinion.) Also, NSF proposals are reviewed by a panel of (mostly) academics drawn from all over the place, who are supposed to be impartial. At Google, the reviewers are both researchers and software engineers who may or may not be working in areas related to the proposal itself.

Most of the proposals I saw left something to be desired. Far too many of them were asking for money for things that we already know how to do (like write an Android app for your pet project) or which have been done to death (like develop yet another mobile ad hoc routing protocol). On the other hand there were the rare proposals that had an exciting new idea and proposed to do something that Google was not about to go do on its own.

A few tips if you ever apply to this program (and I certainly encourage you to do so).

First, think about who the reviewers are. They are mostly not people like me. Most of the engineers at Google don't have a lot of experience reading and reviewing research proposals (let alone writing them), and many of them are not going to be in your immediate research area. Try to reach out and explain why your work is important at a broader level; the reviewers in this case are typically not your peers.

Second, think about why Google should fund this research. The key question I asked myself was not whether Google would benefit from the research, but rather why should Google fund a university to do this project, rather than just do it ourselves. If Google can hire a couple of engineers to solve a problem, I don't see any reason for us to fund a university to do it instead. On the other hand, if the university PIs are going to do something hard, or groundbreaking, or risky that Google would not have the time or resources to do, we should fund it.

There's also the related question of why should Google fund a research effort rather than another funding agency, such as the NSF. This one is a lot easier to answer: I know from experience that it's damned hard to get money from the NSF for many kinds of projects, and Google can help seed new research efforts that would be difficult to get off the ground otherwise. But if a project seems like it can and should be funded through another agency, that makes it less attractive from my perspective.

Third, try to get the exciting ideas up front. Most Googlers are extremely busy and probably won't spend as much time reading the proposals as you'd like them to. If you bury the lead it will be much harder for the reviewers to see the big idea and get excited about your work. It also helps to establish your credentials in the proposal itself -- not just your CV, but a paragraph or two in the main text saying who you are and why you are the right person to do this research is incredibly helpful (especially in the case when the reviewer is outside your area).

Finally, it always helps to have a champion at Google. If there is someone within the company that you know personally, who can vouch for your work and wants to work with you on a project, this helps tremendously. Having a grad student spend time at Google as an intern is a great way to make those connections.

It's just a guess, but I would not be surprised if other companies' research grants worked in much the same way. While I was at Harvard, I got a lot of funding through places like Microsoft Research, Intel, IBM, and Sun, all of which have fantastic university research programs. (Well, except for Sun, which no longer exists.) Of course, keep in mind that I don't speak for the rest of the Google Research Awards committee, and other reviewers very likely use different criteria than I do.

This is my personal blog. The views expressed here are mine alone and not those of my employer.

The death of Intel Labs and what it means for industrial research

2011-04-03T16:15:00.000-07:00

Intel recently announced that it is closing down its three "lablets" in Berkeley, Seattle, and Pittsburgh. I know a lot of people who work at the Intel Labs and in fact spent a year at the Berkeley lab before joining Harvard in 2003. (I should be clear that not all of Intel Research is closing down -- just the lablets.) All of the researchers have been told to find new jobs, though some of them are getting picked up by Intel-sponsored research centers at the nearby Universities.

The Intel Labs were a fantastic experiment to rethink how industrial research should be done. They first started in 2001 under the model that full-time Intel researchers would work side-by-side with faculty and students from the nearby universities. All of the research was done under an open intellectual property model where results were co-owned by the university and Intel. In fact the labs were not inside of the Intel corporate network and operated largely autonomously from the rest of Intel. This allowed projects to be done seamlessly across the Intel/academic barrier and for students to come and go without restrictions on the IP.

Some fantastic work came out of the Labs. The Berkeley lab drove most of the early work on sensor networks and TinyOS, especially while David Culler and his various students were there. The Seattle lab developed PlaceLab (the precursor to WiFi based localization found in every cell phone platform today); WISP (the first computing platform powered by passive RFID); and lots of great work on security of wireless networks. The Pittsburgh lab did work on camera-based sensor networks, cloud computing, and robotics. All of these projects have benefitted tremendously from the close ties that the Labs had with the university.

Before the Labs opened, Intel Research was consistently ranked one of the lowest amongst all major technology companies in terms of research stature and output. I feel that the Labs really put Intel Research on the map by involving world-class academics and doing high-profile projects. They have attracted some of the top PhDs and offered a much more academic alternative to a place like, say, IBM Research.

I have no idea why Intel decided to close the labs. The official press release is devoid of any rationale, and obviously tries to spin the positive angle (the establishment of the university-based research centers which will replace the Labs). I've spoken with a number of the researchers there since the announcement, and have formed my own theories about why Intel is shutting them down. The most obvious possibility is that the Labs are incredibly expensive to run, and it's hard to link the work they do to Intel's bottom line. After all, very little of the work done at the Labs is picked up by Intel's product groups. The Labs' mission has always been to inform the five-to-ten-year roadmap for the company. It's unclear to me whether they have been successful in this, though at least they have inspired some entertaining commercials.

Personally, I'm worried about what this means for industrial computer science research. Here is one of the world's largest and most wealthy tech companies, closing down a set of labs that employs some of the top minds in the field, which by all measures has been really successful in producing novel and high-impact research. If Intel can't figure out how to leverage that amazing talent pool, it does not bode well for the rest of the industry.

Maybe this suggests is that the conventional industrial research model is simply broken. The only (important) places left that use this model are Microsoft, IBM, and HP. These companies can afford to set up big labs with lots of PhDs and pay them to do whatever the hell they want with little accountability, but maybe this model is no longer sustainable. As I've written before, Google takes a very different approach, one in which there is no division between "research" and "engineering." The advantage is that it's always clear how the research activities relate to the company's priorities, although it does mean that researchers are not doing purely "academic" work, the main output of which is more papers.

One closing thought. Perhaps Intel realizes it can have far more impact by setting up large, high-impact research programs within universities rather than run its own labs. In some ways I can appreciate this point of view: help the universities do what they do best. But the way this is being done is unlikely to be successful. The first such Intel center on visual computing involves something like 25 PIs spread across eight universities. Each PI is only getting enough to fund work they were already doing, so this is an example of doing something that looks good on paper but is unlikely to move the needle at all for these research groups. This seems like a missed opportunity for Intel.

Obligatory disclaimer: This is my personal blog. The views expressed here are mine alone and not those of my employer.

Carriers are not ready for tablets

2011-03-23T20:07:00.000-07:00

This week I spent at least two hours on the phone trying to convince both AT&T and Verizon to give me online access to accounts I set up for tablets that I am testing -- the Samsung Galaxy Tab (AT&T) and Motorola Xoom (Verizon). They are both great devices; I like the Galaxy Tab's form factor (like a paperback book) and the Xoom is incredibly fast. But it is clear that the wireless carriers have no idea how to incorporate these devices into their billing and customer service ecosystem. It was such a painful and frustrating experience that I wonder how the cellular carriers expect to leverage these devices as more tablets come onto the market.

First, my story with AT&T. I bought the Galaxy Tab a few months ago which came with an AT&T SIM card pre-installed. When you boot the device for the first time, there's a widget which takes you to a registration page, which I filled out to activate the tablet on AT&T's network. Since then I have not received a bill (to my knowledge) for the usage, and couldn't remember whatever password I might have used to set up the device. Nothing on AT&T's website seemed to offer any help, as it is completely oriented towards phones.

Finally, I gave up and called AT&T to re-register the Tab manually. The first customer service rep had no idea how to do this. They kept asking for the phone number, which the device does not have (at least when you enter the Settings menu it lists the phone number as "unknown"). Given that I had not received any bills I suspected the Tab was never registered, so we had to look it up by IMEI number. The rep could not pull up any account information. She ended up transferring me to technical support at Samsung, of all places. The Samsung rep was very friendly but couldn't help with this problem, either -- it seemed to be an AT&T issue (and I agree). I ended up having to call AT&T back, and went through the same painful process of explaining what my problem was. This rep ended up transferring me over to a different sales rep who tried to help me set up the account from scratch.

This is when things started to go downhill. All I wanted as an unlimited (or as close as possible) data plan for the Galaxy Tab. I could see online that AT&T has a 2GB/month data plan for tablets but the rep kept telling me that "their internal systems don't necessarily show what is on the website." (First warning sign there.) Eventually he managed to pull up the right plan but couldn't seem to figure out how to add a Galaxy Tab -- the device wasn't showing up in his menus. It sounded like he had never activated a tablet before. After around 20 minutes on hold he managed to figure it out, so I think I finally have the Galaxy Tab set up for data access. I was promised that I would get an email from AT&T confirming the new account, but it never arrived. So I guess I am going to have to call them back. I am dreading this.

Verizon was almost as bad. Like the Tab, I had set up the Xoom using the registration app on the tablet itself. I had made a note of the username used to set up the account, but not the password. Verizon's website offers no ability to request a new password except via SMS to the device -- and the Xoom doesn't receive SMS messages, since it's not a phone. The only way to request a new password is to spend around 45 minutes on the phone with various Verizon reps with the result being that a new password is being sent to me by postal mail in five business days. (What is this, the nineteenth century?) Of course, since I moved recently, my mailing address on file with Verizon was incorrect. Fortunately, the service rep allowed me to change the postal address over the phone -- meaning that they trust me enough to let me change my mailing address, not not enough to reset my online account password. This makes absolutely no sense and seems designed to drive users away.

The lesson here is that the wireless carriers have no clue how to incorporate tablets. They are treating them like phones, which they aren't.

Thinking back on 8 years in Boston

2011-03-16T21:59:00.000-07:00

Tomorrow I will be packing up and moving from Boston to Seattle with my family. I thought now would be a good time to reflect on living in Boston as a city and recall some of my best memories here.

I've never lived in Seattle, though have been there many times -- it seems like a wonderful city, full of funky crazy people and absolutely beautiful geography. I'm not terribly excited about the rainy weather, though something tells me it can't be any worse than the Boston winters, when I always feel cooped up. I really miss getting out to go hiking with the dog or mountain biking during the winter months in New England -- and now that I have a kid it's especially hard to get out when it's well below freezing outside (he has a lot lower tolerance for the winter weather than I do). Rain I can deal with; negative 20 wind chills and a foot and a half of snow are something different altogether.

On finishing grad school at Berkeley in 2002, I had a few faculty job offers, and my wife was looking for residency programs in psychiatry. Our decision came down to two cities: Boston, where I had an offer at Harvard, and Pittsburgh, where I had an offer at CMU. I had lived in Boston for a while during college, so I knew I liked the city. But CMU was a very tempting offer, being a much more highly-ranked CS program than Harvard. My wife and I visited Pittsburgh a couple of times and actually liked it a lot: it was a very friendly place, and the CMU folks went all out to show us a good time. At one point we actually made the decision to move to Pittsburgh but decided to sleep on it. The next day we had to ourselves, without anyone showing us around. The only Mexican place that served "fish tacos" appeared to be Van De Kamp's fish sticks on a tortilla. I could not find a music store that allowed you to browse the CDs without an attendant unlocking a glass case to let you inside. We tried to find a decent shopping mall, hoping they would have a good record store, but found ourselves in the mall where the movie Dawn of the Dead was filmed -- I did not make it more than three paces beyond the door before we realized it was a terrible mistake. I'm sorry, Pittsburgh might be a wonderful city for some people, but it was not for us.

Monroeville Mall on a typical Sunday afternoon.

So we moved to Boston in 2003. We drove across the country in my little beat-up two-door Ford, stopped at places like Moab and St. Louis and Nashville, a little like On the Road in reverse. We arrived on a hot, sticky summer day in a thunderstorm and moved into an apartment on Dana Street in Cambridge, not far from Harvard Square. Almost immediately we felt like outsiders. Boston is a very old city, and it shows -- the old brick buildings around Harvard, the beat-up sidewalks, everything dripping with history and significance. Paul Revere. George Washington. Boston Common. Old Granary Burial Ground. The feeling was so different than the newness that is so pervasive in California. It took some getting used to.

These gravestones have been here for a while. (From http://www.flickr.com/photos/harvardavenue/66097831/)

That first summer was one of the most exciting in Red Sox history. I had never paid attention to baseball before, but it soon became clear that if I wasn't conversant in Curt Schilling or Manny Ramirez I was going to be left out of a lot of conversations. Like a lot of people in Boston, I got caught up in the excitement of the 2003 postseason when the Sox were narrowly beat by the Yankees for the ALCS title. The next year was even more exciting, in that the Sox went on to win the World Series for the first time in 86 years. The whole city went totally apeshit. For so many people living here, it was like a moment of rapture that they never believed would actually happen, the culmination of a lifetime of inferiority to cities like New York. I feel sad for some long-time Bostonians who now have no excuse to be cynical about their station in life.

People were just a little excited. (From http://www.flickr.com/photos/eandjsfilmcrew/412103968/)

Not long after we moved to Boston they finally banned smoking in bars and restaurants (which we'd been used to in California) and actually allowed alcohol sales on Sundays. It was as if the city were modernizing before our eyes. It would take another six years for Cambridge to allow outdoor dining at restaurants, which is still encumbered with backwards vestiges of the Puritans (you can only have a drink while sitting outside if you also order food).

Going out in Boston was a bit more a dressy, formal affair than we were used to in Berkeley. At all but the very highest end restaurants in San Francisco, jeans and a t-shirt were acceptable attire; not so in Boston. On the positive side, Boston has a great foodie scene. At first we were totally lost trying to find good places to eat; Zagat's is useless and Yelp simply reflects the lowest common denominator. At first we were convinced that people in Boston had no idea what real, good Mexican food was -- those greasy platters of "enchiladas" covered in melted cheese that are so popular at hellholes like Casa Romero are not it. Then we discovered Chowhound, and got turned on to a whole world of hole-in-the-wall places serving authentic Mexican and Salvadorean and Sichuan and Cambodian. Unlike Berkeley, where you can swing a dead cat and hit three burrito joints and a place with out-of-this-world sushi, in Boston it takes a bit more digging, but there is great food here.

Some of my best memories of living in Boston...

Walking my dog, Juneau, to work at Harvard every day, stopping at the coffee shop on the way, and taking her to the dog park on the way home for an hour or so of play time with the other dogs.

Juneau would occasionally help me reviewing papers, too.

Sitting outside on a warm summer evening, firing up the grill, having friends over for dinner and drinks until late.

Every single fall, feeling the first day of cold air and getting excited for the leaves to start changing.

This image has not been enhanced.

Riding my bike along the banks of the Charles River, whizzing by rollerbladers and clueless tourists walking four abreast in the middle of a bike path.

The morning after a big snowstorm, seeing the world transformed and noticing how quiet everything was with the snow on the ground.

Shoveling is always fun too.

Late nights out in Boston Chinatown with Gu, drinking beer and eating Korean food.

Watching the sun rise out of the window of Mount Auburn hospital on a hot July day a couple of hours before my son was born.

I was a dad not long after this picture was taken.

Running a successful program committee

2011-03-11T17:31:00.000-08:00

Yesterday we held the program committee meeting for the 13th Workshop on Hot Topics in Operating Systems (HotOS), for which I am serving as the program chair. This is the premier workshop in the OS community and focuses on short (five page) position papers meant to bring out exciting new research directions for the field. In some years it has been more exciting than others. What tends to happen is that people send five-page versions of an SOSP submission they are working on, which (in my opinion) is not the best use of this venue. When HotOS becomes an SOSP preview I think it misses an important opportunity to discuss new and crazy ideas that would not make it into a regular conference.

We accepted 33 out of 133 submissions. The number of accepted papers is a bit higher in previous years because I wanted to be more inclusive, but also recognized that we could fit more presentations in at the workshop when you don't have 25-minute talk slots. There is no reason a 5-page paper needs a 25-minute talk, and I think it goes against the idea of the workshop to turn it into a conventional conference.

This was, by far, the best program committee experience I ever had, and I was reflecting on some of the things that made it so successful.

I've been on a lot of program committees, and sometimes they can be a very painful experience. Imagine a dozen (or more) people crammed into a room, piles of paper and empty coffee cups, staring at laptops, arguing about papers for 9 or 10 hours. Whether a PC meeting goes well seems to be the result of many factors...

Pick the best people. We had a stellar program committee and I knew going in that everyone was going to take the job very seriously. Everyone did a fantastic job and wrote wonderful and thoughtful reviews. These folks were invested in HotOS as a venue, were the kind of people who often submit papers to the workshop, and care deeply about the systems community as a whole. The discussion at the PC meeting itself was great, nobody seemed to get cranky, and even after 8+ hours of discussing papers there was still a lot of energy in the room. This is helped a lot by the content -- HotOS papers tend to be more "fun" and since they are so short, you can't nitpick every little detail about them.

Set expectations. I tried to be very organized and made sure that the PC knew what was expected of them, in terms of getting reviews done on time, coming to the PC meeting in person, what my philosophy was for choosing papers, and how we were going to run the discussion at the meeting. I think laying out the "rules" up front helps a lot since it keeps things running smoothly. I've blogged about this before but I think establishing some ground rules for the meeting is really useful.

Get everyone in the room. Having a face-to-face PC meeting is absolutely key to success. Everyone came to the PC meeting in person, except for one person whose family fell ill at the last minute and had to cancel, but even he phoned in for the entire meeting (I can't imagine being on the phone for more than eight hours!). I made sure the PC knew they were expected to come in person, and nailed down the meeting date very early, so everyone was able to commit. Letting some people phone in is a slippery slope. I can't count how many PC meetings I've been to that have been hampered by painful broken conference call or Skype sessions.

Use technology. We used HotCRP for managing submissions and reviews, which is by far the best conference management system out there. During the PC meeting itself, I shared a Google spreadsheet with the TPC which had the paper titles, topic area, accept/reject decision, and a one-line summary of the discussion. The summary was really helpful for remembering what we thought about a paper when revisiting it later in the day. Below is a snippet (with the paper numbers and titles blurred out). The "order" column below is the order in which the paper was discussed. This way, everyone in the PC could see the edits being made in real time and there was rarely confusion about which paper we were discussing next.

Pre-reject and pre-accept. I rejected around half of the submissions before the PC meeting and gave the PC a chance to revive any such paper for discussion (none were). I also "pre-accepted" about 10 papers that were noncontroversial; we saved discussion of these for the end of the day, since they were easy cases. We ended up discussing a total of 69 papers at the meeting, which meant we had to go at a pretty good clip.

Be definitive. With very few exceptions, we tried to reach a clear accept/reject decision on each paper as we discussed it, and did not table any papers for later discussion. There was one case where we were hung on what to do with a paper and decided to push the discussion until the end of the day. In cases where there was disagreement, I would mark a paper as "presumed reject" or "presumed accept" and put down the name of the person who wanted to argue for the opposite outcome later. That gave us a chance to move on when there was an insurmountable debate, and it was clear that the champion (or anti-champion) of a paper would have a chance to have their say.

Take everyone out to a nice dinner afterwards. As far as I'm concerned, this was the best part of hosting the PC meeting.

What life was like before the Web

2011-02-24T19:23:00.000-08:00

It's kind of shocking that the Web has only been around since the mid-1990s, but a lot of younger people that I work with have no idea what it was like to use the Internet before the Web. I'm not that much of an old-timer, but I thought it would be amusing to talk about the pre-HTTP Internet. A few reminisces of the good old days...

Everything was text-based;
There was (almost) no spam;
There was no such thing as a search engine;
You did everything using these crappy UNIX text-based command-line tools.

I first started using the Internet back in high school, around 1990, on an IBM RT UNIX system connected via dialup from school. Back then, there were three main uses of the Internet: email, USENET, and FTP. Email was not very common but some universities had it, and by the time I started college in 1992, you automatically got an email address as an undergraduate.

USENET

USENET was a huge distributed newsgroup system. It was based on peer-to-peer file exchange well before we called it that. There were thousands of newsgroups on topics ranging from the C programming language to the rock band Rush. Yes, it still exists; I think it's largely overrun with spam and warez these days, and I haven't looked at it in years. At the time, spam was almost unheard of so newsgroup discussions tended to stay on-topic. I was the moderator for comp.os.linux.announce for a while, which meant that every announcement to the Linux community (like a new kernel release or software package port) came to my inbox for approval before I posted it to the world.

At some point I want to write a book about USENET culture circa 1992. It was a very interesting place. Groups like talk.bizarre were frequented by the likes of Roger David Carrasso (who pulled off some of the best trolls imaginable); Kibo (founder of alt.religion.kibology); and who can forget the utterly brilliant and bizarre stories by RICHH?

I was a member of the "inner circle" for a group called alt.fan.warlord, which was centered on making fun of ridiculous signatures at the end of USENET posts, like this:

     Paul Tomblin, Head             _            _   ____
     Automated Test Tools Team     | |          | | |  __|   ___._`.*.'_._
    _______________   ______   ____| |________  | |_| |__   +  * .o   u.* `
   /  ________  _  \ |  __  | /  ________  _  \ |  ______|  . ' ' |\^/|  `.
   | |  | |  / / | | | |  | | | | |  |  / / | | | | | |            \V/
   | |__| | / /__| |_| |  | | | |_|  | / /__| |_| | | |            /_\
   \ _____/ \__________|  |_|  \___|_| \__________| |_|       === _/ \_ ===
   //
   \\____   Phone: (613) 723-6500x8018      Mail: Gandalf Data Limited
   /  _  \  Fax:   Don't know it yet              130 Colonnade Road South
   | |_| |  Email: ptomblin@gandalf.ca            Nepean, Ontario
   \_____/   or    ab401@freenet.carleton.ca      K2E 7J5 CANADA

There was a group called alt.hackers that was a moderated group with no moderator. In order to post you needed to figure out how to circumvent the moderation mechanism (which was simply a matter of adding an extra header line to your post).

FTP

USENET was all about discussions, although there were newsgroups where you could post binary files -- typically encoded in an ASCII format like UUENCODE, and broken up into a dozen or more individual posts that you would have to manually stitch back together and decode. This became a popular way to post low-resolution porn GIFs, but was pretty much useless for anything larger than a few megabytes. A better way to download files was to use FTP, which allowed you to download (and upload) files to a remote FTP server. Of course, FTP had this totally unusable command-line interface which required you to type a bunch of commands just to get one file. This site at Colorado State helpfully explains how to use FTP, like anybody still needs to know how. A typical FTP session looked like this:

%   ftp cs.colorado.edu  
Connected to cs.colorado.edu.  
220 bruno FTP server (SunOS 4.1) ready.  
Name (cs.colorado.edu:yourlogin): anonymous  
331 Guest login ok, send ident as password.  
Password:  
230-This server is courtesy of Sun Microsystems, Inc.  
230-  
230-The data on this FTP server can be searched and accessed via WAIS, using  
230-our Essence semantic indexing system.  Users can pick up a copy of the  
230-WAIS ".src" file for accessing this service by anonymous FTP from  
230-ftp.cs.colorado.edu, in pub/cs/distribs/essence/aftp-cs-colorado-edu.src  
230-This file also describes where to get the prototype source code and a  
230-paper about this system.  
230-  
230-  
230 Guest login ok, access restrictions apply.  
ftp> cd /pub/HPSC  
250 CWD command successful.  
ftp> ls  
200 PORT command successful.  
150 ASCII data connection for /bin/ls (128.138.242.10,3133) (0 bytes).  
ElementsofAVS.ps.Z  
   . . .
execsumm_tr.ps.Z  
viShortRef.ps.Z  
226 ASCII Transfer complete.  
418 bytes received in 0.043 seconds (9.5 Kbytes/s)  
ftp> get README  
200 PORT command successful.  
150 ASCII data connection for README (128.138.242.10,3134) (2881 bytes).  
226 ASCII Transfer complete.  
local: README remote: README  
2939 bytes received in 0.066 seconds (43 Kbytes/s)  
ftp> bye  
221 Goodbye.

All this just to download a single README file from the site.

In order to use FTP, you needed to know what FTP sites were out there and manually poke around each one of them to see what files they seemed to host, using "cd" and "ls" commands in the crappy command-line client. There was no such thing as a search engine. So, people in the FTP community made this giant "master list" of every FTP site out there and a short (one-line) summary of what the side had, e.g., "Linux, UNIX utils, GIFs." This master list was mirrored on a bunch of sites but of course that was a manual process, and in effect the mirrors were often conflicting and out-of-date. These days you can find a giant list of FTP sites on the Web, of course.

The Birth of the Web

Back in 1994 or so I was doing research as an undergraduate at Cornell. At the time I was a huge USENET junkie, and from time to time I would see these funny things with "http://" in people's signatures. I had no idea what they were, but at some point saw a USENET post that explained that if you wanted to browse those funny "URL" things you needed to download something called Mosaic from an FTP site at NCSA. When you launched the Mosaic Web browser it brought up this page which was the home page for the entire World Wide Web. At some point I managed to get the original NCSA Web server running and put Cornell's CS department on the Web. Here's an archive of the Cornell Robotics and Vision Lab web page that I made back around 1995.

Not long after this a startup from Stanford called Yahoo created a (manual) index of every web page -- still not quite searchable, but at least you could find things. I remember using the original Google when it was hosted at Stanford and had a whopping "25 million pages" indexed.

Does Google do "research"?

2011-01-22T11:29:00.000-08:00

I've been asked a lot by folks recently about whether the work I'm doing now at Google is "research" and whether one can really have a "research career" at Google. This has also led to a lot of interesting discussions about what the role of research is in an industrial setting. TL;DR -- yes, Google does research, but not like any other company I know.

Here's my personal take on what "research" means at Google. (Don't take this as any official statement -- and I'm sure not everyone at Google would agree with this!)

They don't give us lab coats like this, though I wish they did.

The conventional model for industrial research is to set up a lab populated entirely by PhDs, whose job is mostly to write papers, and (in the best case) inform the five-to-ten year roadmap for the company. Usually the "research lab" is a separate entity from the product side of the company, and may even be physically remote.

Under this model, it can be difficult to get anything you build into production. I have a lot of friends and colleagues at places like Microsoft Research and Intel Labs, and they readily admit that "technology transfer" is not always easy. Of course, it's not their job to build real systems -- it's primarily to build prototypes, write papers about those prototypes, and then move on to the next big thing. Sometimes, rarely, a research project will mature to the point where it gets picked up by the product side of the company, but this is the exception rather than the norm. It's like throwing ping pong balls at a mountain -- it takes a long time to make a dent.

But these labs aren't supposed to be writing code that gets picked up directly by products -- it's about informing the long-term strategic direction. And a lot of great things can come out of that model. I'm not knocking it -- I spent a year at Intel Research Berkeley before joining Harvard, so I have some experience with this style of industrial research.

From what I can tell, Google takes a very different approach to research. We don't have a separate "research lab." Instead, research is distributed throughout the many engineering efforts within the company. Most of the PhDs at Google (myself included) have the job title "software engineer," and there's generally no special distinction between the kinds of work done by people with PhDs versus those without. Rather than forking advanced projects off as a separate “research” activity, much of the research happens in the course of building Google’s core systems. Because of the sheer scales at which Google operates, a lot of what we do involves research even if we don't always call it that.

There is also an entity called Google Research, which is not a separate physical lab, but rather a distributed set of teams working in areas such as machine learning, information retrieval, natural language processing, algorithms, and so forth. It's my understanding that even Google Research builds and deploys real systems, like Google’s automatic language translation and voice recognition platforms.

(Update 23-Jan-2011: Someone pointed out that Google also has a "Quantitative Analyst" job role. These folks work closely with teams in engineering and research to analyze massive data sets, build models, and so forth -- a lot of this work results in research publications as well.)

I like the Google model a lot, since it keeps research and engineering tightly integrated, and keeps us honest. But there are some tradeoffs. Some of the most common questions I've fielded lately include:

Can you publish papers at Google? Sure. Google publishes hundreds of research papers a year. (Some more details here.)You can even sit on program committees, give talks, attend conferences, all that. But this is not your main job, so it's important to make sure that the research outreach isn't interfering with your ability to do get "real" work done. It's also true that Google teams are sometimes too busy to spend much time pushing out papers, even when the work is eminently publishable.

Can you do long-term crazy beard-scratching pie-in-the-sky research at Google? Maybe. Google does some crazy stuff, like developing self-driving cars. If you wanted to come to Google and start an effort to, say, reinvent the Internet, you'd have to work pretty hard to convince people that it could be done and makes sense for the company. Fortunately, in my area of systems and networking, I don't need to look that far out to find really juicy problems to work on.

Do you have to -- gulp -- maintain your code? And write unit tests? And documentation? And fix bugs? Oh yes. All of that and more. And I love it. Nothing gets me going more than adding a feature or fixing a bug in my code when I know that it will affect millions of people. Yes, there is overhead involved in building real production systems. But knowing that the systems I build will have immediate impact is a huge motivator. So, it's a tradeoff.

But doesn't it bother you that you don't have a fancy title like "distinguished scientist" and get your own office? I thought it would bug me, but I'm actually quite proud to be a lowly software engineer. I love the open desk seating, and I'm way more productive in that setting. It's also been quite humbling to work side by side with these hotshot developers who are only a couple of years out of college and know way more than I do about programming.

I will be frank that Google doesn't always do the best job reaching out to folks with PhDs or coming from an academic background. When I interviewed (both in 2002 and in 2010), I didn't get a good sense of what I could contribute at Google. The software engineering interview can be fairly brutal: I was asked questions about things I haven't seen since I was a sophomore in college. And a lot of people you talk to will tell you (incorrectly) that "Google doesn't do research." Since I've been at Google for a few months, I have a much better picture and one of my goals is to get the company to do a better job at this. I'll try to use this blog to give some of the insider view as well.

Obligatory disclaimer: This is my personal blog. The views expressed here are mine alone and not those of my employer.

Fair Harvard

2011-01-13T09:55:00.000-08:00

I've been reflecting on my reasons for moving to Harvard seven years ago. Although I have decided to leave academia, Harvard is really a wonderful place, and I would certainly recommend it to anyone considering an academic career. Back in 2002, I had job offers from several other (big name) universities -- including CMU -- but I chose Harvard for a whole host of reasons, and am really glad that I did. So I thought it would be good to share some of the things that I love about the place.

The campus.

http://farm2.static.flickr.com/1374/5180938910_e483062647_b_d.jpg

This is an easy one. The Harvard campus is just unbelievably beautiful. It feels like the quintessential university: old brick buildings, vines, little walking paths crisscrossing the quads. Even better is that it's not isolated: it is right in the heart of Cambridge, and as soon as you leave campus you're in the middle of Harvard Square which has tons to do. Every day I would take my dog for a walk around campus at lunchtime just soaking in the atmosphere and bumping into Korean tour groups snapping photos of the John Harvard statue.

The Computer Science building.

http://www.flickr.com/photos/grinnell/3293758260/

Maxwell Dworkin Hall is home to the Computer Science and Electrical Engineering faculties. It was built in 1999 and named for Bill Gates' and Steve Ballmer's mothers -- no joke. It's one of the best CS buildings I've been to on any university campus (and I have visited a lot). The faculty offices are huge, there are great common spaces, and the overall layout (with a central open staircase) promotes interaction. It is a great place to work.

The faculty.

http://news.harvard.edu/gazette/story/2010/03/inside-electronic-commerce/

Probably the number one reason I joined Harvard was to interact with all of the incredible faculty. The thing that struck me the most when I interviewed was that the CS faculty were an amazing bunch that really have their act together and were all doing incredibly interesting things. Although a lot of places talk about being cross-disciplinary, Harvard embraces it like no other place that I've seen. David Parkes (pictured above) and Yiling Chen work across computer science and economics; Krzysztof Gajos on the boundary of AI and HCI; Radhika Nagpal across biology, systems, and algorithms; Stephen Chong and Greg Morrisett across systems and languages; Hanspeter Pfister across graphics, systems, and scientific computing ... among many other examples. What's great about this is that the faculty are not isolated in their own little worlds -- since there is so much cross-fertilization, everyone knows a lot about what the other faculty are doing. In a smaller department this is absolutely essential.

The students.

http://photos.cs50.net/Events/CS50-Fair-2010-Hugon/

This is another easy one. I have raved about Harvard students on this blog before, but it's worth mentioning again. (One of my favorite students of all time, Rose Cao, pictured above.) Working with and teaching Harvard students has been one of the highlights of my career. They are smart, engaged, creative, passionate about what they do. It is extremely rare to meet a student who was just "along for the ride" or trying to coast through college -- not at Harvard. They pushed me and challenged me to do better all the time. My graduate students were similarly amazing, an extremely dedicated bunch who were not afraid to take risks. Although Harvard's a smaller school, I was never for want of great grad students to work with.

The department culture.

http://picasaweb.google.com/radpicasa/peoplessr#

One thing that is hard to get right in any university setting is the overall culture and feel of the department. If it's broken, it's incredibly hard to fix. Harvard's CS department feels like a family. There's a sense of a shared mission to make the place better and everyone pulls their weight. For junior faculty, it is a really supportive environment: the senior faculty work hard to make sure that everyone has the opportunity to be successful. One of the most remarkable things is how decisions get made -- nearly always by consensus, after a lot of discussion (generally during faculty lunch meetings). There's very little "department politics" and all voices are heard. Harvard chooses to hire new faculty who will fit into this culture, so there's a really strong emphasis on retaining this structure, which is great.

The size.

http://www.flickr.com/photos/mexbox/4171583189/

One of the main reasons I came to Harvard, rather than to a much larger department, was to have impact. I've often compared being at Harvard to being at a startup (though an incredibly well-funded one) instead of, say, a huge company like IBM (or Google for that matter). Though I feel that Harvard needs to hire a few more CS faculty to attain critical mass across all areas, I really like the smaller department feel of the place. Pretty much all of the faculty can fit in a single room for a lunch meeting and have a substantive discussion. Each individual faculty member can have a tremendous amount of impact on teaching and research. So being at Harvard was a great opportunity to shape an important CS department and this is one of the things that really attracted me to the place.

The city.

http://farm4.static.flickr.com/3196/3091615359_fba3419d01_b.jpg

Finally, there is a lot to love about Cambridge and Boston. It's a beautiful city and very compact. I could walk to work every day in about 20 minutes, and along the way pass countless restaurants, bars, coffee shops, record shops, you name it. Cambridge has a great mix of "big city" and "residential" feeling to it -- it is kind of like an oversized college town. Over the years I have amassed an impressive list of places to eat and things to do around the city and it never gets old. OK, I'll admit that the weather is not always ideal, but it's probably my favorite city on the East Coast, and a lot more livable (and affordable) than New York.

What would I change?

I don't want to be too negative, but for balance I think I should mention two things that I would change about Harvard, if I had the chance. The first is that the tenure process is too damn long. If all goes according to schedule, the decision is made at the end of your seventh year, which keeps you in limbo for quite a long time, making it hard to get on with your life. On the other hand, the good thing about a long tenure clock is that you can take big risks (which I did) and have time to make course corrections if necessary. Harvard is not unique in having a long tenure clock (as I understand it, CMU's is longer!), but still long compared to the more typical four or five year clocks.

The second thing is that the CS faculty really needs to grow by a few more faculty in order to realize its full potential. I think they need to hire at least five or six new faculty to reach critical mass. As I understand it there are plans to hire over the next few years, which is good.

Looking back, I'm really glad that I went to Harvard as a new faculty member. It's hard to imagine that I would have had anywhere near as much impact had I gone somewhere else.

The Best Things about 2010

2011-01-04T17:18:00.000-08:00

Last year I posted the Best Things About 2009, so I feel compelled to do the same this year. In what is sure to become an annual tradition, I present to you the Best Things About 2010 -- Volatile and Decentralized edition.

Best portrayed cameo appearance in a major Hollywood motion picture:

Brian Palermo's dramatic and riveting 45-second performance as myself in The Social Network. The rest of the movie is just okay, but the scene where I am teaching virtual memory to Mark Zuckerberg is one of the most compelling moments in modern cinema, right up there with Daniel Day-Lewis in the final scene of There Will Be Blood.

Best phone call:

It was early June and I was sitting by the pool in Providenciales, Turks and Caicos Islands, drinking a rum punch, when my cell phone rang. "Hi Matt, it's Greg." Morrisett, that is, at the time the CS department chair at Harvard. "Oh, hi Greg," I said, nonchalantly, as though I was used to getting phone calls from him while being thousands of miles away. "I have good news," he said, and told me that my tenure case had just been approved by the President. I believe my exact response was, "no shit!" (in the surprised and delighted sense, not the sarcastic sense). In that moment I felt that seven years of hard work had finally paid off.

Best album:

$O$ by Die Antwoord. Every now and then an album comes along that I get utterly addicted to, and listen on repeat for days on end until I'm sick of it... and then I listen some more. Die Antwoord's unbelievable mashup of white-boy rap, totally sappy techno, and over-the-top lyrics in English, Afrikaans, and Xhosa is one such album. This is not at all the kind of music I usually listen to, but something about the trashy hooks and ridiculous vocals is just too catchy. The best song is "Evil Boy," which has a brilliant video (warning: definitely NSFW). Also check out the faux documentary video "Zef Side." Runners up: Transference by Spoon; Root for Ruin by Les Savy Fav.

Best reason to memorize Goodnight Moon and "Elmo's Song:"

Being daddy to an eighteen-month-old. Fatherhood continues to wear well on me. As my little boy, Sidney, gets older, he only manages to be more amazing and more entertaining. These days he's running, talking, singing, parroting back pretty much anything he hears (gotta watch what I say around him), coloring with crayons, counting, naming everything in sight. It's also exhausting taking care of him at times, but totally worth it in every way.

Day in the Life of a Googler

2010-12-26T12:02:00.000-08:00

I was thinking recently about how different my workdays are now that I'm at Google, compared to the faculty job at Harvard. The biggest difference is that I spent nearly 90% (or more) of my time writing code, compared to Harvard where I was lucky if I got half an hour a week to do any programming. I also spend a lot less time at Google procrastinating and reading a zillion stupid websites -- mostly because I'm enjoying the work a lot more.

Here's a short rundown of my typical day at Google:

6:30am - Wake up, get son up, shower, breakfast, take dog to the park.

8:30am - Leave for work (I take the subway most days).

9:00am - Arrive at work. Type passwords into half a dozen different windows to get my work environment back to a sane state. Check email. Check on status of my several jobs running in various datacenters. Page in work from day before.

9:30am-10:15am - Work on code to add requested feature to the system I'm working on. Debug it until it's working, write a unit test or two. Fire off code changelist for review. Grab third free Diet Coke of the day.

10:15-11:00 - Switch git branches to another project. Take a look at code review comments from a colleague. Go through the code and address the comments. Build new version, re-run tests, re-run lint on the code to make sure it's working and looks pretty. Submit revised changelist and responses to comments.

11:00-11:30 - Switch git branches again. Rebuild code to be safe, then fire off a three-hour MapReduce job to crunch log data to analyze network latencies.

11:30 - 12:00 - Quick videoconference meeting with team members in Mountain View.

12:00-12:35 - Lunch of free yummy food in cafeteria. Regale coworkers with stories of Apple IIgs hacking when I was in middle school.

12:35-2:00 - Back at desk. Check email. Check status of MapReduce job - about halfway done. Respond to last set of comments from code review done in the morning and submit the code. Merge and clean up the git branch. Take a look at task list to decide what to work on next.

2:00-3:00 - Project meeting with teams in Cambridge, Mountain View, and elsewhere by videoconference. This is my only hour-long meeting of the whole week. It is mildly amusing and I mostly spend the time doing some light hacking on my laptop and hitting reload on the MapReduce status page to see if it's done yet. Check Buzz and post a snarky comment or two.

3:00-4:00 - Red Bull infusion to keep energy going for the rest of the day. MapReduce is finally done. Generate graphs of the resulting data and stare at them for a while. Think about why the results are different than expected and write next version of code to generate another set of statistics. Try to get the code to the point where I can fire off another MapReduce before leaving for the day.

4:00-5:00 - Whiskey Thursday! Round up a group of colleagues to drink scotch and play Guitar Hero. (I have a nice collection of scotch under my desk. Somehow I have been designated as the guardian of the alcohol supply, which suits me fine.)

5:00 - Pack up laptop and head home.

5:30-8:00 - Dinner and family time until son goes to bed.

8:00 until bedtime - More hacking, if there's stuff I want to get done tonight, or make a few nice cocktails if not.

Contrast this to my typical work day at Harvard:

6:30am - Wake up, get son up, shower, breakfast, take dog to the park

8:30am - Leave for work (a 20-minute walk from home to the office, and I bring the dog with me).

9:00am - Arrive at office. Check email. Groan at the amount of work I have to do before the onslaught of meetings in the afternoon.

9:15am - Start working on outline for a grant proposal. About three minutes later, decide I don't know what I want to write about so spend next 45 minutes reading Engadget, Hacker News, and Facebook instead.

10:00am - Try to snap out of the Web-induced stupor and try to make headway on a pile of recommendation letters that I have to write. Fortunately these are easy and many of them are cut-and-paste jobs from other recommendation letters I have written for other people before.

11:00am - Check calendar, realize I have only an hour left to get any real work done. Respond to some emails that have been sitting in my inbox for weeks. Email my assistant to set up three more meetings for the following week.

11:30am - Try to make some token headway on the grant proposal by drafting up a budget and sending off the three emails to various support staff to get the paperwork going. Make up a title and a total budget for the proposal that sound reasonable. Still undecided on what the project should be about.

12:00pm - Take dog out for a 20-minute walk around campus. Sometimes spend longer if we run into other dogs to play with.

12:30pm - Run over to Law School cafeteria to grab overpriced and not-very-appetizing lunch, which I eat sullen and alone in my office, while reading Engadget and Hacker News.

1:00pm - First meeting of the day with random person visiting from a random company in Taiwan who will never give me any money but wants me to spend half an hour explaining my research projects to them in extraordinary detail.

1:30pm - Second meeting of the day with second-semester senior who has suddenly decided after four aimless years in college that he wants to do a PhD at Berkeley or MIT. Explain that this will not be possible given zero research track record, but somehow end up promising to write a recommendation letter anyway. Mentally note which other recommendation letters I will cut and paste from later.

2:00pm - Realize that I have to give lecture in half an hour. Pull up lecture notes from last year. Change "2009" to "2010" on the title slide. Skim over them and remember that this lecture was a total disaster but that I don't have time to fix it now.

2:30pm - 4:00pm - Give lecture on cache algorithms to 70 or so somewhat perplexed and bored undergrads. Try to make the lecture more exciting using extensive PowerPoint animations and wild gesticulations with the laser pointer. Answer a bunch of questions that remind me why the lecture was a disaster last year and vow to fix it before delivering again next year.

4:00-4:10pm - Hide in office with door closed trying to calm down after adrenaline rush of lecturing. Gulp large amounts of Diet Coke to re-energize and re-hydrate.

4:10-4:20pm - Check email. Check Engadget. Check Facebook.

4:30-5:00pm - Last meeting of the day with two grad students working on a paper due in less than a week. They have no outline and no results yet but are very optimistic that they will make it in time. Spend half an hour sketching ideas and possible graphs on the whiteboard while they scribble furiously in their notebooks. Make vague promises about reviewing a draft if I see one later in the week.

5:00pm - Walk home with my dog. This is the best part of my day.

5:30pm - Get home, immediately sit down to check enormous pile of email that accumulated while I was in lecture and meetings. Forward five new meeting requests to my assistant for scheduling next week.

5:45pm - 8:00pm - Family time, dinner.

8:00pm - Pretend to "work" by reading email and tinkering with PowerPoint slides for a talk I have to give the next week. Too exhausted to do anything useful, make a drink and read Engadget again.

Guest Post: Why I'm staying at Harvard (by Michael Mitzenmacher)

2010-11-16T18:27:00.000-08:00

[Michael Mitzenmacher is a professor of Computer Science and the Area Dean for Computer Science at Harvard. He is a dear friend and colleague and has been one of the role models for my own career. Michael wanted to respond to my earlier blog post on leaving Harvard with his own reasons for staying; I am only too happy to oblige. (I swear I did not ghost write this.) You can read more of Michael's own blog here, though he's not posting much these days. --MDW]

To begin, I'd like to say how sorry we are at Harvard that Matt's not returning. Matt's been a great colleague, continually pushing to make CS at Harvard better. His enthusiasm and tenaciousness have made us tangibly better in numerous ways. I, personally, will miss him a lot. Matt pushes hard for what he believes in, but in my experience he's always done so with open ears and an open mind. We're losing a leader, and Google is lucky to have him. I have no doubt he'll do great things for the company, and maybe even earn them another billion or two.

While Matt's decision has been a blow to CS at Harvard, I'm optimistic that our plan for growth will, eventually, make up for that loss. My job as Area Dean is to try to make that happen as soon as possible. I don't want to suggest that replacing Matt will be easy, but rest assured we'll be on the case.

I'd also like to say that I think I understand Matt's reasons for leaving. I'm glad to have him write "I love Harvard, and will miss it a lot." And how could I disagree with statements like "The computer science faculty are absolutely top-notch, and the students are the best a professor could ever hope to work with. It is a fantastic environment, very supportive, and full of great people." But I know from previous talks with him that he hasn't always loved being a professor. And that's what I'll try to write about the rest of the post.

I think there's a sense in academia that people get PhD's so that they can become professors. Most graduate students have that point of view going in -- their experience with research professionals at that point is essentially entirely with faculty. And most professors encourage students to have that goal. Some of that, I think, is that most professors like their job (unsurprisingly), and some may not have other experiences to suggest to their students. And some of it may be more calculated. One measure of a faculty member's success is how many faculty offspring they've produced.

But being a faculty member is not for everyone. As Matt has described in this blog, and I in the past have described in my blog, being a professor is probably not exactly what most people expect. Besides teaching and research, your time gets taken up with administration, managing (graduate) students, fundraising, and service to your scientific community. It's perhaps absurd to expect that everyone who starts out in a PhD program be interested in all these various aspects of the job. And, fortunately, in computer science, there are still many other compelling options available.

As Matt says, at Google, "I get to hack all day." That's just not true as a faculty member -- time for actual hacking is usually pretty small, and more of your time is spend managing others to hack for you. (This is a complaint I've heard from many faculty members.) I can understand why Google would be a very appealing place for someone who wants to write code. I'm sure Matt will come to miss some of the other aspects of being a professor at some point, and I'd imagine Google will to some extent let him entertain some of those aspects.

One of the comments suggested money must be a motivation. For some people who have to make this choice, maybe it is. (See Matt's comments on the post below for his take on that.) So what? Again, it's good that in our field there are good options that pay well. That's a big plus for our field, especially if we accept the fact that not everyone can be or wants to be a professor. But as Matt says, professors at Harvard (and top 20 institutions in general) are doing just fine, and money probably isn't the main issue for those who choose a different path.

I suppose the question that's left is why I'm staying at Harvard -- that is, why I still like being a professor. (And thank you to those of you who think the obvious answer is, "Who else would hire you?") I enjoy the freedom of working on whatever I find interesting; being unrestricted in who I choose to talk to about research problems and ideas; having the opportunity to work with a whole variety of interesting and smart people, from undergraduates to graduate students to CS colleagues all over the globe to math and biology professors a few buildings down; the ample opportunity to do consulting work that both pays well and challenges me in different ways; the schedule that lets me walk my kids to school most every day and be home for dinner most every night; and the security that, as long as I keep enjoying it, I can keep doing this job for the next 30+ years.

The job is never boring. On any given day, I might be teaching, planning a class, working with students, thinking, writing a paper, writing some code, reading, listening to a talk, planning or giving a talk, organizing an event, consulting in some form, or any other manner of things. In the old days, I wrote a blog. These days, I'm administrating, making sure our classes work smoothly, our faculty are satisfied and enabled to do the great things they do, and we're able to continue to expand and get even better. Once I wrote a book, and someday I hope to do that again. Perhaps the biggest possible complaint is that there's always something to do, so you have to learn to manage your time, say no, and make good decisions about what to do every day. As someone who hates being bored, this is generally a good feature of the job for me.

And Harvard, I find, is an especially great place to work. We attract some of the most amazing students. Our still small-ish CS faculty really works together well; we all know who each other are, we keep aware of what we're all doing research-wise, we collaborate frequently, and we compromise and reach consensus on key issues. Outside of the CS faculty, there's all sorts of interesting people and opportunities on campus and nearby. Boston is a great city (albeit too cold and snowy in the winter).

Other profs have made similar comments in Matt's post -- there's a lot to like about the job, and at the same time, it's not the best choice for everyone. Of course I don't like everything about the job. Getting funding is a painful exercise, having papers rejected is frustrating and unpleasant, and not every student is a wondrous joy to work with. I sometimes struggle to put work away and enjoy the rest of my life -- not because of external pressure (especially post-tenure), but because lots of my work is engaging and fun. Of course that's the point -- there's good and bad in all of it, and people's preferences are, naturally, vastly different. I don't think anyone should read too much into Matt's going to Google about the global state of Computer Science, or Professordom, or Harvard, or Google. One guy found a job he likes better than the one he had. It happens all the time, even in academia. It's happened before and will happen again.

But I'm happy with my job right now. In fact, I'm pretty sure my worst day on the job this year was the day Matt told me he wasn't coming back. We'll miss you, Matt, and best of luck in all your endeavors.

Why I'm leaving Harvard

2010-11-15T07:33:00.000-08:00

The word is out that I have decided to resign my tenured faculty job at Harvard to remain at Google. Obviously this will be a big change in my career, and one that I have spent a tremendous amount of time mulling over the last few months.

Rather than let rumors spread about the reasons for my move, I think I should be pretty direct in explaining my thinking here.

I should say first of all that I'm not leaving because of any problems with Harvard. On the contrary, I love Harvard, and will miss it a lot. The computer science faculty are absolutely top-notch, and the students are the best a professor could ever hope to work with. It is a fantastic environment, very supportive, and full of great people. They were crazy enough to give me tenure, and I feel no small pang of guilt for leaving now. I joined Harvard because it offered the opportunity to make a big impact on a great department at an important school, and I have no regrets about my decision to go there eight years ago. But my own priorities in life have changed, and I feel that it's time to move on.

There is one simple reason that I'm leaving academia: I simply love work I'm doing at Google. I get to hack all day, working on problems that are orders of magnitude larger and more interesting than I can work on at any university. That is really hard to beat, and is worth more to me than having "Prof." in front of my name, or a big office, or even permanent employment. In many ways, working at Google is realizing the dream I've had of building big systems my entire career.

As I've blogged about before, being a professor is not the job I thought it would be. There's a lot of overhead involved, and (at least for me) getting funding is a lot harder than it should be. Also, it's increasingly hard to do "big systems" work in an academic setting. Arguably the problems in industry are so much larger than what most academics can tackle. It would be nice if that would change, but you know the saying -- if you can't beat 'em, join 'em.

The cynical view is that as an academic systems researcher, the very best possible outcome for your research is that someone at Google or Microsoft or Facebook reads one of your papers, gets inspired by it, and implements something like it internally. Chances are they will have to change your idea drastically to get it to actually work, and you'll never hear about it. And of course the amount of overhead and red tape (grant proposals, teaching, committee work, etc.) you have to do apart from the interesting technical work severely limits your ability to actually get to that point. At Google, I have a much more direct route from idea to execution to impact. I can just sit down and write the code and deploy the system, on more machines than I will ever have access to at a university. I personally find this far more satisfying than the elaborate academic process.

Of course, academic research is incredibly important, and forms the basis for much of what happens in industry. The question for me is simply which side of the innovation pipeline I want to work on. Academics have a lot of freedom, but this comes at the cost of high overhead and a longer path from idea to application. I really admire the academics who have had major impact outside of the ivory tower, like David Patterson at Berkeley. I also admire the professors who flourish in an academic setting, writing books, giving talks, mentoring students, sitting on government advisory boards, all that. I never found most of those things very satisfying, and all of that extra work only takes away from time spent building systems, which is what I really want to be doing.

We'll be moving to Seattle in the spring, where Google has a sizable office. (Why Seattle and not California? Mainly my wife also has a great job lined up there, but Seattle's also a lot more affordable, and we can live in the city without a long commute to work.) I'm really excited about the move and the new opportunities. At the same time I'm sad about leaving my colleagues and family at Harvard. I owe them so much for their support and encouragement over the years. Hopefully they can understand my reasons for leaving and that this is the hardest thing I've ever had to do.

SenSys 2010 in Zurich

2010-11-07T06:29:00.000-08:00

Photo from http://www.flickr.com/photos/aforero/542248140

I just got back from Zurich for SenSys 2010. I really enjoyed the conference this year and Jan Beutel did a fantastic job as general chair. The conference banquet was high up on the Uetliberg overlooking the city, and the conference site at ETH Zurich was fantastic. We also had record attendance -- in excess of 300 -- so all around it was a big success. I didn't make it to all of the talks but I'll briefly summarize some of my favorites here.

Sandy Pentland from the MIT Media Lab gave a great keynote on "Building a Nervous System for Humanity." He gave an overview of his work over the years using various sensors and signals to understand and predict people's behavior. For example, using various sensors in an automobile it is often possible to predict in advance whether someone is about to change lanes, based on subtle prepatory movements that they make while driving. His group has also used wearable sensors to gather data on conversational patterns and social interactivity within groups, and used this data to study practices that influence a business' productivity. This was an amazing keynote and probably the best we have ever had at SenSys -- very much in line with where a lot of work in the conference is headed.

The best paper award on Design and Evaluation of a Versatile and Efficient Receiver-Initiated Link Layer for Low-Power Wireless was presented by Prabal Dutta. This paper describes a new MAC layer based on receiver initiation of transmissions: receivers send probe signals that are used to trigger transmissions by sending nodes with pending packets. Their approach is based on a new mechanism called backcast in which senders respond to a receiver probe with an ACK which is designed to constrictively interfere with multiple ACKs being transmitted by other sending nodes. This allows the receiver probe mechanism to scale with node density. Because A-MAC does not rely on receivers performing idle listening, cross-channel interference (e.g., with 802.11) does not impact energy consumption nearly as much as LPL.

There were a bunch of talks this year on use of cell phones and other sensors for participatory sensing applications. One of my favorites was the paper on AutoWitness from Santosh Kumar's group at the University of Memphis. In this work, a small tag is embedded within a high-value item (like a TV set). If the item is taken from the home, accelerometer and gyro readings are used to determine its probable location. Using HMM-based map matching they showed that they can reconstruct the path taken by a burglar with fairly high accuracy.

Chenyang Lu from WUSTL presented a paper on Reliable Clinical Monitoring using Wireless Sensor Networks: Experience in a Step-down Hospital Unit. This paper presents one of the first studies to make use of low-power wireless sensors in a real hospital environment with real patients. My group spent about seven years working on this problem and we were often frustrated at our inability to get medical personnel to sign on for a full-scale study. Chenyang's group managed to monitor 46 patients in a hospital over 41 days (but only three patients at a time). Their paper showcases a lot of the challenges involved in medical monitoring using wireless sensors and is a must-read for anyone working in the area.

Finally, Steve Dawson-Haggerty from Berkeley presented his work on sMAP, a framework for tying together diverse sensor data for building monitoring. Steve's observation is that while different companies have worked on various protocols for standardizing building monitoring applications, most of these systems are highly proprietary, vertically-integrated nightmares of multiple entangled protocols. Steve took a "Web 2.0" approach to the problem and designed a simple REST-based API permitting a wide range of sensors to be queried through a Web interface. This is a really nice piece of work and demonstrates what is possible when a clean, open, human-centric design is preffered over a design-by-committee protocol spec with twenty companies involved.

Speaking of companies, one disappointing aspect of this years' conference is that there were very few industrial participants. None of the papers were from companies, and only a couple of the demos had any industrial affiliation. Part of the reason for this is that the conference organizers didn't approach many companies for support this year, since the budget was adequate to cover the meeting expenses, but this had the negative effect of there being essentially zero industrial presence. My guess is that the companies are going to the IEEE sensor nets conferences, but I am deeply concerned about what this means for the SenSys community. If companies aren't paying attention to this work, we run the risk of the wheels of innovation grinding to a halt.

There was one talk this year that was highly controversial -- Tian He's group from University of Minnesota presented a paper on an "energy distribution network" for sensor nets. The idea is to allow sensor nodes to push energy around, in this case, using wires connecting the nodes together. Unfortunately, the presenter did not justify this design choice at all and the only experiments involved very short (1 meter) cables between nodes. It seems to me that if you connect nodes together using wires, you can centralize the power supply and bypass the need for a low-power node design in the first place. The fact that the presenter didn't have any good arguments for this design suggests that the research group has not spent enough time talking to other people about their work, so they've built up a herd mentality that this actually makes sense. I don't think it does but would love to hear some good arguments to the contrary.

Apart from SenSys, I had the chance to (briefly) visit Timothy Roscoe at ETH Zurich as well as connect with some colleagues at the Google Zurich office. ETH Zurich is a very exciting place: lots happening, lots of faculty growth, tons of resources, good students and postdocs. I was very impressed. Even more impressive is Google's office in Zurich, which has the most over-the-top design of any of the Google offices I've visited so far (including Mountain View). The office is beautifully laid out and has a bunch of humorous design touches, including an indoor jungle and firepoles that connect the floors (with a helpful sign that reads, "don't carry your laptop while sliding down the pole.")

Conference talk pet peeves

2010-11-03T07:31:00.000-07:00

I'm sitting here at SenSys 2010 in Zurich and listening to some pretty interesting -- and also some pretty dull -- talks on the latest research in sensor networks. Now seems like an appropriate time for a blog post I've been saving for a while -- some of the things that really annoy me when I'm listening to a talk. Of course, I'm sometimes guilty of these myself, and I'm not the best speaker either. But I guess I have license to gripe as a listener.

There are lots of tips on there on how to give a good talk. David Patterson's "How to give a bad talk" is a great summary of what NOT to do. Some of these things are fairly obvious, like not cramming too much text on one slide, but others I see happen again and again when I'm listening to talks at a conference.

The dreaded outline slide: Nearly every 25-minute talk in a systems conference has the same format. Why do speakers feel compelled to give the mandatory outline slide --

"First, I'll give the motivation and background for this work. Next, I'll describe the design of FooZappr, our syetem for efficient frobnotzing of asynchronous boondoggles. Next, I'll describe the implementation of FooZappr. Then, I will present evaluation, and finally, related work and conclusions..."

After having seen several hundred such talks I have this memorized by now, so I don't think it is a good use of time. An outline slide is sometimes a good idea for a longer talk, but it should have some content -- guideposts for the audience, or highlights of the major ideas. This is rarely needed for a short conference talk.

Reading the slides: The number one thing that drives me up the wall is when the speaker simply reads the text on the slide, or essentially says the same thing in slightly different words than what is printed on the bullets. This is lazy, and suggests that the talk hasn't been rehearsed at all. It's also the fastest way to bore the audience. Most members of the audience have two parallel reception channels: visual and auditory -- so I try to use both at once and provide (slightly) redundant information across the two channels in case of loss (e.g., tuning out the slide).

No sense of design: It can be physically painful to watch an entire talk crammed full of multiple fonts, clashing colors, inconsistent use of graphics, and that awful PowerPoint clip art (you know the ones: skinny stick figures scratching their heads). Modern presentation software, including PowerPoint, lets you design beautiful and visually compelling talks -- use it! If you insist on coming up with your own template, at least use the colors and fonts in a minimal and consistent way. I tend to use the Harvard crimson banners on my slides and the same color for highlight text. A grad student once complemented me on the beautiful font choice in my talk -- it was Helvetica. You shouldn't spend too much time on this, after all, if your slides look good but have terrible content, it's not worth it.

No sense of humor: I've lost count of how many conference talks I've heard that are nothing more than dry recitations of the technical content of the paper. No attempt is made at humor anywhere in the talk - not a joke to warm up the audience, or at least a visual joke somewhere in the slides to wake people up a bit. A conference talk is entertainment (albeit an obscure kind of entertainment for an incredibly dorky audience) -- the speaker should at least make some effort to make the talk interesting and delightful. Most conference attendees spent hundreds of dollars (thousands if you include travel) for the privilege of listening to your talk, so you owe it to them to deliver it well. This is not to say that you should overload the talk with jokes, but breaking up the presentation with a bit of levity never hurt anyone.

Keep in mind that a conference talk is meant to be an advertisement for your paper. You do not have to cram every technical detail in there. What will the audience remember about your talk? I'll never forget Neil Spring's talk on ScriptRoute where he used a bunch of ridiculous custom Flash animations.

Of course, the talk delivery matters tremendously. If you're one of those dull, monotonic speakers or have a thick accent, you are probably not going to get a reputation as a good speaker. If you sound totally bored by your talk, the audience will be too. Some grad students are surprised that this matters so much and think it shouldn't -- but if you're planning on pursuing an academic career, you have to give a LOT of talks. So you should get good at it.

"Let's take that offline." This is a frequent response to a question that the speaker doesn't want to answer. I've heard speakers jump immediately to this rather than make any attempt whatsoever of answering. This has become far too socially acceptable at conferences and I think questioners (and session chairs) should push back. It is occasionally OK to take a discussion offline if it is going to be a lengthy discussion or there's clearly no agreement between the speaker and questioner, but I think speakers should be expected to answer questions posed after the talk.

Finally, with digital cameras there's an increasing trend of audience members taking photos of every talk slide (sometimes for every talk). Here at SenSys there's someone who is taking a video of every talk on his cell phone. I find this fairly obnoxious, especially when the photographer insists on using a flash and leaving on the camera's shutter "beep". If you want my slides, just ask me and I'll send you the PPT. I also think it's rude to take a video of a talk without asking the speaker's permission.

Computing at scale, or, how Google has warped my brain

2010-10-19T18:48:00.000-07:00

A number of people at Google have stickers on their laptops that read "my other computer is a data center." Having been at Google for almost four months, I realize now that my whole concept of computing has radically changed since I started working here. I now take it for granted that I'll be able to run jobs on thousands of machines, with reliable job control and sophisticated distributed storage readily available.

Most of the code I'm writing is in Python, but makes heavy use of Google technologies such as MapReduce, BigTable, GFS, Sawzall, and a bunch of other things that I'm not at liberty to discuss in public. Within about a week of starting at Google, I had code running on thousands of machines all over the planet, with surprisingly little overhead.

As an academic, I have spent a lot of time thinking about and designing "large scale systems", though before coming to Google I rarely had a chance to actually work on them. At Berkeley, I worked on the 200-odd node NOW and Millennium clusters, which were great projects, but pale in comparison to the scale of the systems I use at Google every day.

A few lessons and takeaways from my experience so far...

The cloud is real. The idea that you need a physical machine close by to get any work done is completely out the window at this point. My only machine at Google is a Mac laptop (with a big honking monitor and wireless keyboard and trackpad when I am at my desk). I do all of my development work on a virtual Linux machine running in a datacenter somewhere -- I am not sure exactly where, not that it matters. I ssh into the virtual machine to do pretty much everything: edit code, fire off builds, run tests, etc. The systems I build are running in various datacenters and I rarely notice or care where they are physically located. Wide-area network latencies are low enough that this works fine for interactive use, even when I'm at home on my cable modem.

In contrast, back at Harvard, there are discussions going on about building up new resources for scientific computing, and talk of converting precious office and lab space on campus (where space is extremely scarce) into machine rooms. I find this idea fairly misdirected, given that we should be able to either leverage a third-party cloud infrastructure for most of this, or at least host the machines somewhere off-campus (where it would be cheaper to get space anyway). There is rarely a need for the users of the machines to be anywhere physically close to them anymore. Unless you really don't believe in remote management tools, the idea that we're going to displace students or faculty lab space to host machines that don't need to be on campus makes no sense to me.

The tools are surprisingly good. It is amazing how easy it is to run large parallel jobs on massive datasets when you have a simple interface like MapReduce at your disposal. Forget about complex shared-memory or message passing architectures: that stuff doesn't scale, and is so incredibly brittle anyway (think about what happens to an MPI program if one core goes offline). The other Google technologies, like GFS and BigTable, make large-scale storage essentially a non-issue for the developer. Yes, there are tradeoffs: you don't get the same guarantees as a traditional database, but on the other hand you can get something up and running in a matter of hours, rather than weeks.

Log first, ask questions later. It should come as no surprise that debugging a large parallel job running on thousands of remote processors is not easy. So, printf() is your friend. Log everything your program does, and if something seems to go wrong, scour the logs to figure it out. Disk is cheap, so better to just log everything and sort it out later if something seems to be broken. There's little hope of doing real interactive debugging in this kind of environment, and most developers don't get shell access to the machines they are running on anyway. For the same reason I am now a huge believer in unit tests -- before launching that job all over the planet, it's really nice to see all of the test lights go green.

In Defense of Mark Zuckerberg

2010-10-10T19:53:00.000-07:00

I finally got to see The Social Network, the new movie about the founding of Facebook. The movie is set during my first year teaching at Harvard, and in fact there is a scene where I'm shown teaching the Operating Systems course (in a commanding performance by Brian Palermo -- my next choice was Brad Pitt, but I'm thrilled that Brian was available for the role). The scene even shows my actual lecture notes on virtual memory. Of course, the content of the scene is completely fictional -- Mark Zuckerberg never stormed out of my class (and I wouldn't have humiliated him for it if he had) -- although the bored, glazed-over look of the students in the scene was pretty much accurate.

It's a great movie, and very entertaining, but there are two big misconceptions that I'd like to clear up. The first is that the movie inaccurately portrays Harvard as a place full of snobby, rich kids who wear ties and carry around an inflated sense of entitlement. Of course, my view (from the perspective of a Computer Science faculty member) might be somewhat skewed, but I've never seen this in my seven years of teaching here. Harvard students come from pretty diverse backgrounds and are creative, funny, and outgoing. I've had students from all corners of the world and walks of life in my classes, and I learn more from them than they'll ever learn from me -- the best part of my job is getting to know them. I've only seen one student here wearing a tweed jacket with elbow patches, and I'm pretty sure he was being ironic.

The second big problem with the movie is its portrayal of Mark Zuckerberg. He comes across in the film as an enormous asshole, tortured by the breakup with his girlfriend and inability to get into the Harvard Final Clubs. This is an unfair characterization and not at all the Mark Zuckerberg that I know. The movie did a good job at capturing how Mark speaks (and especially how he dresses), but he's nowhere near the back-stabbing, ladder-climbing jerk he's made out to be in the film. He's actually an incredibly nice guy, super smart, and needless to say very technically capable. If anything, I think Mark was swept up by forces that were bigger and more powerful than anyone could have expected when the Facebook was first launched. No doubt he made some mistakes along the way, but it's too bad that the movie vilifies him so. (Honestly, when I first heard there was a movie coming out about Facebook with Mark Zuckerberg as the main character, I couldn't believe it -- the quiet, goofy, somewhat awkward Mark that I know hardly sounded like a winning formula for a big-budget Hollywood film.)

The take-away from the movie is clear: nerds win. Ideas are cheap and don't mean squat if you don't know how to execute on them. To have an impact you need both the vision and the technical chops, as well as the tenacity to make something real. Mark was able to do all of those things, and I think he deserves every bit of success that comes his way. As I've blogged about before, I once tried to talk Mark out of starting Facebook -- and good thing he never listened to me. The world would be a very different (and a lot less fun, in my opinion) place if he had.

Harvard Computer Science is hiring

2010-09-24T06:40:00.000-07:00

We are hiring new faculty in CS this year at Harvard - applications encouraged! See the job posting below.

Tenure-track Professor in Computer Science
Harvard University

The Computer Science program at Harvard benefits from its outstanding undergraduate and graduate students, an excellent location, significant industrial collaboration, and substantial support from the Harvard School of Engineering and Applied Sciences.

We invite applications for a position as a tenure-track professor in Computer Science. The appointment is expected to begin on July 1, 2011.

We welcome outstanding applicants in all areas of computer science. We are particularly interested in areas related to machine learning, probabilistic modeling, and artificial intelligence. In terms of applications, areas of interest include computational science, engineering, or the social sciences. We encourage applications from candidates whose research examines computational issues raised by very large data sets or massively parallel processing.

Candidates should have an outstanding research record and a strong commitment to undergraduate teaching and graduate training. Applicants must have completed a Ph.D. by September 1, 2011. Information about Harvard's current faculty, research, and educational programs is available at http://www.seas.harvard.edu/teaching-learning/areas/computer-science

Candidates should send, ideally as a single PDF document, a curriculum vitae, a list of publications, a statement of research and teaching interests, and up to three representative papers to the following email address: cs-search@seas.harvard.edu. In addition, candidates should have at least three letters of reference sent to the above address.

Alternatively, material may also be sent via surface mail to:

CS Search Committee
Harvard School of Engineering and Applied Sciences
Harvard University
Maxwell Dworkin 153
33 Oxford Street
Cambridge, MA 02138

Applications will be reviewed as they are received. For full consideration, applications should be received by December 1, 2010.

Harvard is an Equal Opportunity/ Affirmative Action Employer.
Applications from women and minority candidates are strongly encouraged.

Getting started as a PhD student

2010-09-18T11:43:00.000-07:00

Since it's the beginning of the semester, I've been thinking a bit about the common advice that I give to incoming PhD students. Say you're a new PhD student in Computer Science. What are the main things you should know when getting started? Here are some of my favorite tidbits; feel free to chime in with your own in the comments.

(Turns out that Margo Seltzer blogged on the same topic this week too! Great minds think alike.)

This year I have two new students -- Amos Waterland and Youngjune Gwon -- I'm kinda bummed that I'm on sabbatical and can't interact with them as much as I'd like, but they will be busy with classes anyway :-)

Don't let school get in the way of your education. My advisor, David Culler, was fond of this misquote of Mark Twain, but it's true. Classes and program requirements are important but what is far more important is becoming an expert in your area. If that means taking fewer classes each term so you have time to do some real research, so be it. Harvard's PhD program is course-heavy and I often advise my students to ignore the requirements on paper and spread the classes out over several years, taking no more than two "real" classes at a time. Otherwise they would get nothing done for the first couple of years of the program.

Just dive in and have no fear. I often liken starting a research project (or career) to wandering into a dense jungle and blazing a path -- ideally a new path (but sometimes the jungle has overgrown since anyone wandered in your direction). There's only so much you can learn standing outside the jungle looking in, or reading about it, studying it, pondering it, whatever. I also feel that you can't really understand a problem until you've tried to solve it, even if your approach is somewhat naive. So rather than sit around reading a gazillion papers, just dive in and start doing some research -- anything -- even if you think it might be a dead end. Half the time you discover that something you thought would be easy or uninteresting actually leads to a bunch of open problems once you get beyond a superficial understanding of the area.

When I started on my thesis, I spent a lot of time reading and talking and thinking before writing any code, and at one point convinced myself that my project idea was stupid and not worth pursuing. (Whether this is true or not is still open for discussion.) Finally I sat down (during the sessions at OSDI 2000) and pounded out a simple prototype to test my ideas.

Don't read too much (at first). Obviously a huge part of grad school is reading and reviewing papers, but the problem with reading too many papers (especially at first) is that it can make it look like all of the interesting problems have already been solved. I've seen more than one grad student get into a rut because they read too many papers. Certainly you should never read anything from the 1960's or 70's or you will realize that it all has been done before -- by Real Programmers who had to code in assembly on a trinary architecture with sixteen levels of virtual address space segmentation and only two registers -- but I digress.

There is a real advantage to the Zen Mind approach of jumping in with your gut instincts about a problem and not worrying too much if you are treading familiar ground. At some point -- say 2 or 3 months into a project -- you should take a step back and compare your approach to what has come before, and correct course if necessary. Arguably all great systems projects are just reinventing ideas and reevaluating assumptions in light of changing technology trends, but don't let that stop you. I made my career that way, you can too!

Keep track of the papers that you do read. Come up with a good system for tracking the papers you have read, need to read, and take notes on them that you can easily reference later. Printing them out and scribbling in the margins is OK, though there may be more environmentally-friendly approaches. I am a big fan of the Mac application Papers, which is like iPhoto for PDF files. Mendeley, CiteULike, and Bibdesk offer similar functionality. In grad school I just kept a huge text file of every paper I read and my notes on it. When it came time to write my thesis, this was invaluable for putting together the related work list (same for papers that I wrote).

Finally, take a lot of notes. A couple of my students have the habit of coming to meet me empty-handed, which is a problem when I give them pointers to related work or ideas to chase down, which they really should be writing down. (Hell, I'm never going to remember what I told them from one week to the next, so they need to keep track.) I find it really helpful to maintain a group Wiki with meeting notes where I can go back and see what we were talking about week to week.

So, you want to go to grad school?

2010-09-09T04:19:00.000-07:00

Every year I am approached by students asking about grad school in Computer Science. I generally sit down with them for an hour or so and go over all of the details of why you should go, what the tradeoffs are, where you should apply, what it takes to get in, and so forth. I figured it would be a good idea to write some of this advice up in a blog post so I can capture it in a more permanent form.

In this post, I will talk about why to do a PhD in Computer Science, and why not to do a PhD. Assuming you've already decided to go to grad school, I've blogged previously about how to get in. Later on I'll blog about where you should apply.

Masters vs. Ph.D.

First off, when I talk about "grad school," I mean doing a PhD. Many students ask me about doing a Master's degree after college. I don't generally recommend students from good CS programs do a Master's in CS, for several reasons: (1) it's expensive, (2) you can learn the same material as an undergrad, and (3) doing a Master's isn't useful for deciding if you want to do a PhD -- it is a totally different experience. M.S. programs generally require taking a lot of classes, so they are not at all like being a PhD student (where the focus is on research). PhD programs don't generally care whether you have a Master's when you apply; in fact, some schools seem to prefer taking students straight out of their undergrad degree.

The only cases I recommend doing a Masters are for students that aren't quite prepared to get into a top-ranked PhD program, for example, because their undergrad major is in something other than CS. (Note that if your undergrad major is in an area closely aligned with CS, such as engineering, math, or physics, or you took a lot of CS classes despite majoring in something else, you probably don't need a Master's.) A Master's can also benefit students coming from foreign universities. Doing a Master's at a good CS program in the US is a good way of getting a letter from a well-known CS professor to help you get into a PhD program.

Why do a PhD?

Of course, this is the most fundamental question. I'll try to articulate the pros and cons below. First, the pros:

Lots of freedom. PhD-level research is all about defining a problem, solving it, and convincing everybody that your solution is a good one. Half of the challenge of doing a PhD is deciding what problem to work on. It is really about carving out your own niche in the field.
Working for yourself. Once you have a PhD -- and even during the process of getting one -- you are able to be your own boss. Rather than working on someone else's vision, you are the one to define the vision. This is especially true if you pursue an academic career after grad school, but is also the case in many industrial research labs. Typically, people with Bachelor's and Master's degrees aren't afforded so much freedom.
Working on the hardest problems. PhD research is about opening up new avenues of enquiry, and working on problems that the rest of the world hasn't even articulated yet. If you do it right, you can have tremendous impact.

Why not do a PhD?

Of course, doing a PhD is not for everybody. I have seen quite a few students enter a PhD program, spin their wheels for years on end, and leave without finishing their degrees or doing much of anything. I've even see people get a PhD without making a mark on the academic community, just barely doing enough to get a thesis signed off by three professors (this is easier than it sounds). These people shouldn't have done a PhD at all -- they would have been better off going straight to industry, making a lot more money, and probably being much happier in their jobs.

The only reason to do a PhD is because you love doing research. If you don't love research, don't bother -- it is not worth the time, money (in terms of opportunity cost vs. making a real salary in industry), or stress. Doing a PhD is stressful, if you are doing it right -- you are in constant competition with other academics to publish your results in the top venues, to make a name for yourself, to get recognized. If you harbor ideas of lazy days sitting in the coffee shop pondering the universe, you are dead wrong. (You can always approach a PhD this way, but you will probably not be very successful.)

"But," you say, "I don't know if I love research -- I've never done any!" Then why are you considering doing a PhD at all? The only way to find out is by doing research, preferably as an undergrad. If you screwed up and graduated before doing research, try to find a research assistant job in a professor's lab, or do a Master's (see above). Be warned that most Master's programs are very course-intensive, so you will need to work extra hard to do some research on top of the courseload.

Another downside to the PhD is that is it extremely unstructured. This can drive some people crazy. The nature of research is that it is open-ended, and there are often no clear guideposts as to what you should be working on each day. Also, your PhD advisor may or may not mesh with your personality -- they might be too hands-off, too hands-on, out to lunch, too stressed about getting tenure, etc. Your experience in grad school will depend a lot on how well you get along with your advisor. (Let me take this opportunity to apologize to all of my current and former students for what they have to put up with.)

Doing a PhD can take a long time. Nobody finishes in four years. The typical time to completion is around five or six years, but there is a long tail -- I reserve the term "paleo-student" for someone who has been at it more than 10 years. See the Taulbee Survey for some data. The time to finish your degree can be taxing, since all of your friends have already gone ahead and gotten married, had kids, bought a house, etc. while you're still living in squalor with four roommates who haven't bathed in a week. Eventually your parents and loved ones start wondering what the hell you are doing with your life. The brilliant comic strip Piled Higher and Deeper uses this as a recurring theme. My advisor used to say that "doing a PhD costs you a house," which is just about right if you consider the amount of money you could have made being in industry for the same amount of time.

So, should you do a PhD, or not? If you think you are up for it, you can always try it for a couple of years, and if you dislike it, go get a job in industry instead. Unfortunately, it doesn't quite work the other way -- moving from industry to grad school is much harder.

Taking a year off

A lot of students tell me that they plan to get their bachelor's degree, work in industry "for a year or two" and then apply to grad school "later." If you are serious about going to grad school, I do not recommend this approach. In my experience, it is quite rare to make the jump from industry to grad school. First off, industry pays so much better than the PhD student stipend that it is quite hard to make this transition. Also, to get into a top PhD program, you need good letters from CS professors, and letters from industry don't really count. After you've been gone for a couple of years it's hard to get those stellar letters from the professors that may have loved you back when you were in college; newer, brighter, more energetic students have taken your place and you are long forgotten (although maybe Facebook will change all that). Industry experience rarely helps a graduate application, especially if you're some low-level engineer at a big company writing tests all day.

That said, taking time off after college can be a great experience. I took a year off doing research at different universities (University of Cambridge, University of Glasgow, and Vrije Universiteit in Amsterdam) after finishing college but before applying to grad school. It was a great experience and it bolstered my grad school applications since I stayed within the academic sphere.

Another approach is to get into grad school and then defer admission for a year. Most schools will let you do this (although they may grumble a little, or even make you re-apply, although this is usually a formality).

Proposal: The Restaurant Incubator

2010-08-24T17:24:00.000-07:00

The problem: Starting a new restaurant is a huge undertaking, requiring the would-be restaurateur to raise a large amount of capital, find a good location, buy furniture, hire staff, get the word out, etc. All of this overhead severely limits risk-taking in the kitchen since it distracts from the mission of creating great food.

My proposal: Apply concepts from technology startup incubators (such as Y Combinator) to the restaurant industry. Give up-and-coming young chefs the opportunity to focus on cooking and creativity, and leverage shared infrastructure to reduce overheads.

I'm a big fan of Top Chef. (See my earlier proposal for a reality TV show for junior computer science faculty -- Top Prof. Bravo should be calling any minute now...) So naturally I see parallels between what the aspiring young chefs on that show are doing and what tech entrepreneurs face when starting a company. The tech industry has found ways to make it much easier for a new idea to get out into the real world, leveraging technologies such as universal Internet access and cloud computing. Why not apply the same ideas to the restaurant industry?

Here's my concept. Open a restaurant called, say, Restaurant Wars, after the popular Top Chef challenge. On a given night, three or four independent chefs each prepare and serve their own menu to the guests. They share a (large) kitchen, some amount of the ingredients, prep staff, wait staff, front of house, perhaps even the wine list. The space, tables, chairs, china, etc. are all owned by the restaurant. Guests can order from any of the chef's menus and are encouraged to provide feedback after the meal.

Get a big-name chef like Tom Colicchio or Ferran Adrià (he needs something new to do, anyway) to serve as in-kitchen mentor for the chefs.

The owners are investing in the future of the participating chefs and take, say, a 15% ownership in any independent restaurant venture that they launch after participating. Chefs spend up to, say, 3 months at Restaurant Wars, ensuring that there is constant turnover and thereby renewed interest from diners.

Of course there are a couple of kinks to work out (one of which is that my wife thinks this is a really dumb idea). The first is that it's hard to serve radically different styles of cuisine side-by-side. It sets up for some odd comparisons. Also, there needs to be a way to manage food costs across the "competing" chefs; if one is cooking with ridiculously expensive ingredients (say, a terrine of abalone served with a civet-cat coffee foam topped with Beluga caviar ) you need a way to limit costs and keep things equitable. Another is whether potential diners would go for a place with so much turnover in the kitchen, although that's the whole idea. Maybe Tom or Ferran can guest chef one night a month to maintain street cred.

If anyone has $20 million lying around and wants to go in with me on this, drop me a line. I'll be happy to help with the cocktail menu.

Book Review - The Victorian Internet

2010-08-08T17:04:00.000-07:00

I just finished reading The Victorian Internet: The Remarkable Story of the Telegraph and the Nineteenth Century's Online Pioneers by Tom Standage. (I read most of it on my iPhone using the Kindle app.) The book was first published in 1998, and it's a great read about the development and impact of the telegraph. Of course, there are a lot of uncanny similarities between the development of the telegraph and that of the Internet. It's really interesting to imagine what living in a world before the telegraph must have been like: information could only travel as fast as a messenger on a horse, train, or steamship.

The book is not targeted at a technical audience and I was disappointed that there was not enough said about how messages got relayed through the telegraph network -- what was the routing protocol? There is some discussion of different signaling methods and Morse code, of course, as well as the many variations on Morse's telegraph design (including some really far-out designs that included multiplexing multiple operators over a single wire, essentially using TDMA to avoid conflicts).

There is some interesting discussion on the precursor to the electric telegraph, namely optical telegraphs, which amounted to networks of towers placed on hilltops using visual signals to convey information. These were fairly widespread in Europe in the 18th century and in some places it took a while for the electric telegraph to supplant them.

Some interesting tidbits are strewn throughout the book:

A wide range of crazy schemes were devised to compress and encrypt information sent via telegraph, especially for business purposes. This caused problems for telegraph operators who were more prone to introducing errors when keying in unfamiliar strings of letters, and decreased the sending rate as well. At one point the ITU imposed a 15-letter limit on code words and required that they be composed of pronounceable syllables. This led to bogus code words like "APOGUMNOSOMETHA" (I am proud to report that Google offers zero results for this word -- I guess I just Googlewhacked it).
There was a 19th century equivalent of the DNS: in Britain, individuals and companies could reserve a special "telegraphic address" that allowed others to send them a message without knowing their real, physical address. These were assigned on a first-come, first-served basis and each telegraph office had a giant book listing all of the addresses that had been registered.
It took years for the telegraph to be recognized as anything other than a novelty. Morse and others struggled to convince the governments of US and Britain that they should invest in the development and deployment of the telegraph; early demonstrations did not convince U.S. Senators who (obviously) couldn't read strips of paper printed in Morse code.
The original Transatlantic telegraph cable took years to complete, and broke four times while the ships were laying it out. It failed after only a few months of use.
A period of fifty years elapsed between the development of the telegraph and the telephone.

Among many others. It's a good read, short and sweet, and makes me want to outfit my DSL router in an oiled wooden box with brass dials and steam valves, like a good steampunk retrofit.

Proposal: Abolish faculty offices

2010-07-30T11:35:00.000-07:00

Posited: faculty offices are detrimental to the advancement of scientific knowledge.

At Google, everyone sits out in the open at clusters of desks (not cubicles, God no). It looks a little something like this:

(This appears to be a picture from Google's Kirkland, WA office, but we have a similar setup in Cambridge.)

Today I swung by Harvard to my big, empty office, which looks like this:

Of course, it's an awesome office, one of the most spacious that I've seen in an academic CS building. You could easily pack eight grad students in there, sitting on top of a large pile of undergrads.

I got to thinking. In most academic settings, faculty are isolated in their own separate offices -- isolated from one another, from the students, from the rest of the world. This can't possibly be good for cross-fertilization of ideas. Although I leave my office door open whenever I'm there, people hardly ever drop by -- I guess I am pretty intimidating. (Or maybe it's my ferocious guard dog that I bring with me to work.)

Of course, having my own office is great for meetings, but there are plenty of places I could hold meetings instead. And it's nice to have a place for all of my books and journals, but really, shouldn't those be in a communal library anyway? And I guess the office is nice for when I want to shut out the world and try to concentrate, but that's nothing a pair of noise-canceling headphones can't fix.

So here's the idea -- let's get rid of faculty offices. Get everyone sitting together in open-floorplan space, interacting, communicating, innovating. Just like startups. Why not? This is the model that the Berkeley RADLab uses. All of the faculty sit together in an open space. Here's a picture of Randy Katz at his desk in the lab, surrounded by British war paraphernalia:

Doesn't he look happy? (You can read more about the RADLab design philosophy here.)

To be honest, when I started at Google I was pretty concerned about the lack of an office. I was sure that I would be unable to concentrate sitting out in the open, and would get annoyed at all of the distractions and bodily odors of the people around me. On the contrary, I've found that it actually helps my productivity to be in an active space with other people hacking away around me. Also, the noise level is rarely an issue. People are generally respectful and it's a little like working in a coffee shop.

When I get back to Harvard, I think I'll move into the lab with my grad students. (I can hear the groaning now.)

A Retrospective on SEDA

2010-07-26T18:23:00.000-07:00

I keep bumping into references online to my PhD thesis work on the Staged Event-Driven Architecture, or SEDA. I thought this had been long forgotten, but I guess not. It's been about 10 years since I did the bulk of that work (the major paper was published in SOSP 2001), so I thought it would be interesting to think back on what we got right and what we got wrong. Just to summarize, SEDA is a design for highly-concurrent servers based on a hybrid of event-driven and thread-driven concurrency. The idea is to break the server logic into a series of stages connected with queues; each stage has a (small and dynamically-sized) thread pool to process incoming events, and passes events to other stages. The advantages of the design include modularity, the ability to scale to large numbers of concurrent requests, and (most importantly, I think) explicit control of overload, through the queues.

Apparently quite a few systems have been influenced by SEDA, including some major components that drive Google and Amazon. I occasionally hear war stories from folks that tried the SEDA design and abandoned it when the performance did not meet up with expectations. The events-versus-threads debate continues to rage on. See, for example, this recent post comparing the performance of Node.js and Clojure. (Who knew that people would be talking about implementing high-performance servers in JavaScript and LISP? And I thought using Java for SEDA was crazy....)

Some historical context

It's important to keep in mind that I started work on SEDA around 1999. At the time, the server landscape looked pretty different than it does now. Linux threads were suffering a lot of scalability problems, so it was best to avoid using too many of them. Multicore machines were rare. Finally, at the time nearly all papers about Web server performance focused on bulk throughput for serving static Web pages, without regard for end-to-end request latency.

These days, things are pretty different. Linux threading implementations have vastly improved. Multicores are the norm. With the rise of AJAX and "Web 2.0," request latency matters a lot more.

Before we start splitting hairs, I want to emphasize that the SEDA work is about a server architecture, not an implementation. Yes, I implemented a prototype of SEDA (called Sandstorm) in Java, but I never considered Sandstorm to be the main contribution. Unfortunately, a lot of follow-on work has compared C or C++ implementations of alternate server designs to my original Java implementation. It is really hard to draw many conclusions from this, in part because Sandstorm was heavily tuned for the particular JVM+JIT+threading+GC combination I was using at the time. (I spent an incredible amount of time trying to get gcj to be robust enough to run my code, but eventually gave up after around six months of hacking on it.) Probably the best head-to-head comparison I have seen is David Pariag et al.'s paper in EuroSys 2007, where they do a nice job of factoring out these implementation effects.

What we got wrong

In retrospect, there definitely a few things about the SEDA design that I would rethink today.

The most critical is the idea of connecting stages through event queues, with each stage having its own separate thread pool. As a request passes through the stage graph, it experiences multiple context switches, and potentially long queueing at busy stages. This can lead to poor cache behavior and greatly increase response time. Note that under reasonably heavy load, the context switch overhead is amortized across a batch of requests processed at each stage, but on a lightly (or moderately) loaded server, the worst case context switching overhead can dominate.

If I were to design SEDA today, I would decouple stages (i.e., code modules) from queues and thread pools (i.e., concurrency boundaries). Stages are still useful as a structuring primitive, but it is probably best to group multiple stages within a single "thread pool domain" where latency is critical. Most stages should be connected via direct function call. I would only put a separate thread pool and queue in front of a group of stages that have long latency or nondeterministic runtime, such as performing disk I/O. (This approach harkens back to the original Flash event-driven server design that SEDA was inspired by.) This is essentially the design we used in the Pixie operating system.

I was never completely happy with the SEDA I/O interface. My original work on Java NBIO was used as the foundation for Sandstorm's event-driven socket library. (I was also one of the members of the Java Community Process group that defined the java.nio extensions, but I preferred to use my own library since I wrote the code and understood it.) However, layering the SEDA stage abstraction on top proved to be a real pain; there are multiple threads responsible for polling for request completion, incoming sockets, and so forth, and performance is highly sensitive to the timing of these threads. I probably spent more time tuning the sockets library than any other part of the design. (It did not surprise me to learn that people trying to run Sandstorm on different JVMs and threading libraries had trouble getting the same performance: I found those parameters through trial-and-error.) The fact that SEDA never included proper nonblocking disk I/O was disappointing, but this just wasn't available at the time (and I decided, wisely, I think, not to take it on as part of my PhD.)

Of course, while Java is a great implementation language for servers, I didn't implement Sandstorm with much regards for memory efficiency, so it kind of sucks in that regard compared to leaner server implementations.

What we got right

I chose to implement SEDA using Java, in order to tie into the larger Berkeley Ninja project which was all in Java. It turned out that my Java code was beating servers implemented in C, so I saw no reason to switch languages. I still believe that had I tried to do this work in C, I would still be writing my PhD thesis today. Case in point: Rob von Behren, who did a follow-on project to SEDA, called Capriccio, in C, never finished his PhD :-) Never mind -- we both work for Google now.

The most important contribution of SEDA, I think, was the fact that we made load and resource bottlenecks explicit in the application programming model. Regardless of how one feels about threads vs. events vs. stages, I think this is an extremely important design principle for robust, well-behaved systems. SEDA accomplishes this through the event queues between stages, which allow the application to inspect, reorder, drop, or refactor requests as they flow through the service logic. Requests are never "stalled" somewhere under the covers -- say, blocking on an I/O or waiting for a thread to be scheduled. You can always get at them and see where the bottlenecks are, just by looking at the queues. I haven't seen another high performance server design that tries to do this -- they mostly focus on peak performance, not performance under overload conditions, which was my main concern. I also think that SEDA makes it easier to design services that are load aware, though I leave it as an exercise to the reader to determine how you would do it in a conventional thread or event-driven framework.

Honestly, we never took full advantage of this, and I struggled somewhat to come up with a good benchmark to demonstrate the importance of this idea. (When you're using SpecWeb99, you can't really drop or refactor Web page requests.) Benchmarks are tricky, but I think that many real-world services have the opportunity to leverage SEDA's explicit load conditioning model.

Some general comments

I'm not really working on high performance server designs anymore (although my stint at Google may or may not take me back in that direction). I'm also not up on all of the latest literature on the topic, so maybe there is a killer design out there that solves all of these problems once and for all.

One thing I learned doing this work is that one should always be skeptical of simple, "clean" benchmarks that try to demonstrate the peak or best-case performance of a given server design. My original benchmarks of SEDA involved fetching the same static 8KB web page over and over. Not surprisingly, it yields about the same performance no matter what server design you use. This benchmark hardly stresses the I/O, memory, threading, or socket layers of any system, and is more likely to highlight performance differences in the corner cases. (Believe me, I've read plenty of papers that use much dumber benchmarks than this. SpecWeb99, which we used in the SOSP paper, is only marginally better.)

It's harder to do, but I think it's important to evaluate performance in the context of a "real" application, one that involves all of the load and complexity you'd see in a real service. So I am not convinced by microbenchmarks anymore; it is like showing off a new automobile design running on a flat, even, indoor track with no wind drag, no adverse weather, no other traffic, and no need for seatbelts or airbags. Usually as soon as you load it up with realistic conditions, things start to break. Achieving good, robust performance across a wide range of loads is the real challenge.

Fatherhood and professorhood

2010-07-22T17:36:00.000-07:00

My little boy, Sidney, turned a year old this past week. I've been reflecting a lot lately on how much my life has changed since having a baby. I've also met a bunch of junior faculty members who ask what it's like trying to juggle being a prof with being a parent. To be sure, I was pretty worried that it would be really hard to juggle my work responsibilities with having a kid. At first I screwed up royally, but now I've found a good balance and it really works. Best of all, I love being a dad -- it has been worth all the sleepless nights, cleaning up barf and poop, and learning how to steer a spoonful of beets into an unwilling mouth.

Of course, being a dad is totally different than being a mom, and I can't imagine how much harder it must be for women in academia who want to have children. My wife is also in an academic career. When Sidney was first born, she took 3 months off of work, but this was hard for both of us -- for her, because she never got a break from taking care of the baby during the day, and for me, since I wasn't doing a good job at balancing my job with being a new dad. Fortunately, Sidney was born about a week after I submitted my tenure case materials, so I could relax a little, but being a prof still involves a lot of day-to-day stress.

My biggest mistake was not taking teaching relief as soon as the baby was born. I was slated to teach one of our intro courses, which had around 80 students, so it would have been a real problem had the course not been covered that term. I figured since I had taught the class a couple of times before it would be easy -- I planned to lean heavily on the teaching assistants and mostly waltz in twice a week to give lectures I had already prepared. What I didn't account for is that with so many students there is always a fire to put out somewhere -- a student who needs special attention, allegations of cheating, TAs dropping the ball -- so you are still "on call" even if the lectures and assignments have been prepared well in advance. The biggest stressor was having to teach on days without having had any sleep the night before. In retrospect, trying to teach that term was a huge mistake, and I should have put my own sanity before the department teaching schedule.

Since then, things have improved greatly, and I am so happy and proud to be a dad. The thing that nobody tells you is that newborn babies aren't much fun. They can't yet smile, laugh, control their extremities, see more than 6 inches away, or do much of anything except eat, sleep, cry, and poop. Once they hit 10 or 12 weeks things really take a big turn, and now that Sidney is a year old he is a total hoot. He just started walking last week and it's the funniest thing in the world to watch.

The biggest change in my life is that I can no longer work in the evenings and on the weekends. When I'm home, I'm daddy, and finding time to sit down at the laptop to get anything done is pretty hard. After Sidney's 8pm bedtime I can get some things done, but by that time, my two priorities are having a nice cocktail and getting a good nights' sleep. (By the way, I am a big fan of the Ferber method for helping babies learn to sleep on their own. Greg Morrisett described the technique to me as "exponential backoff." We did this with Sidney when he was 4 months old and since then has consistently slept from 8pm - 6am almost every night. It works.)

On the flip side, when I'm in the office, I am very focused on getting work done, since I know I can't work as well in the evenings. So rather than put off things until after dinner, I try to knock them off during the day. As a result I'm a lot more productive and less scattered. I feel like a total slacker leaving the Google office at 5pm sharp every day, but I have to get home to meet the nanny. That's life. Now that Sidney is a little older we've been taking him out to restaurants and happy hour -- there's nothing like feeding the baby a bottle while nursing a nice cold beer of my own. So life is good. Professors can also be parents. I just can't wait to start teaching Sidney C++.

The Amazing Undergrads of Summer

2010-07-16T18:23:00.000-07:00

I've often said that one of the best things about being at Harvard is the students. The undergrads in particular are really out-of-this-world, needle-goes-to-eleven, scary smart. (There's no way I would have ever managed to get into Harvard as an undergrad.) I also love getting undergrads involved in research, and have had some great experiences. Some of my former students have gone off to do PhDs at Berkeley, Stanford, and MIT, off to medical and business school, to work at Microsoft, Amazon, and Google. Others have started little companies, like Facebook. I'm really proud of the students that have passed through my research lab and my classes.

But the batch of undergrads I'm working with this summer are off the charts in so many ways. I'm so excited about what they're doing I feel like I have to brag a little.

Most of them are working on the RoboBees project. In case you haven't heard, this project is Sean Hannity's #1 waste of government stimulus money, and our goal is to build a colony of micro-sized robotic bees. We have a bunch of undergrads involved this summer on a wide range of projects. The last two weeks, I asked them to each give 5-10 minute updates to the group on their status, and expected most of them to say that they hadn't done very much yet. I was totally blown away that each and every one of them has already done more than we expected them to get done in the entire summer. They are kicking total ass. In no particular order:

Matt Chartier '12 is studying the use of RoboBees for exploring underground environments, like mines and collapsed buildings. He's put together a very nice subterranean environment generator for our RoboBees simulator -- it generates very realistic mine tunnels and shafts -- and is looking at different sensors for detecting warm bodies in an underground setting.

Diana Cai '13 has developed a 3D visualization environment for the simulator, by hooking up Java3D and the JBullet physics engine. This thing is sweet -- we can watch the simulated bees fly through the environment, pan the view angle, and change a bunch of other parameters. This is going to be one of the most important tools for this project and Diana has knocked it out of the park. Check out the below movie for an example of it in action.

Lucia Mocz '13 is developing a simulation of the optical flow sensors that we plan to use on the RoboBees platform. Optical flow will allow the RoboBees to navigate, avoid obstacles, and more, and now we can explore how effectively these sensors enable closed-loop control. The last time I talked with Lucia she was geeking out on tuning the gains for her PID control loop for hover control. Keep in mind she just finished her freshman year at Harvard -- I didn't even know what a PID control loop was until grad school!

Peter Bailis '11 is cranking on getting our micro-helicopter testbed up and running, writing TinyOS code to control the sensors, motors, and servos, and making it possible to control a swarm of helis via a Python API from a PC. He's also working on the new distributed OS that we're developing for RoboBees (all very top secret stuff!). Here's a little video of one of our helis taking off and landing (and not crashing) using Peter's code. Today -- one helicopter taking off. Tomorrow -- world domination:

Rose Cao '11 is exploring the use of harmonic radar for tracking RoboBees in the field. The idea is to outfit each bee with a lightweight transponder that reflects radar at a specific frequency which we can detect. Of course, we also need to worry about disambiguating multiple bees which could be done by controlling their flight patterns. Rose also gave the funniest and most creative PowerPoint presentation I've seen in a long time!

Neena Kamath '11 and Noah Olsman '12 (a student at USC, here on the RoboBees REU program) are working with Radhika Nagpal on algorithms for crop pollination, exploring a wide range of themes including random walks, Levy flight patterns, adaptive gradients, and energy awareness. This stuff is super fun and highlights the power of a large colony of RoboBees working together.

Finally, a shout out to my non-RoboBee student, Thomas Buckley '12, who is working on integrating our Mercury wearable sensor platform with LabView to make it possible for non-experts to program and tune the sensor network in different clinical settings. No more hacking NesC code just to change the sampling parameters!

All of these great students are supported by the National Science Foundation, National Instruments, and Harvard's PRISE program for undergraduate research. Thanks to all of them for their support!

The subtle art of managing a research group

2010-07-07T17:29:00.001-07:00

One thing that you rarely learn before starting a faculty job is how much work goes into managing a research group. During my pre-tenure days, this meant squeezing the most productivity out of my students and making sure they were hitting on all cylinders. Now that I have tenure, my role is more like a bodhisattva -- simply to make sure that my students (and postdocs and research staff) are successful in whatever they do. Of course, productivity and success have a high degree of overlap, but they are not exactly the same thing.

There are many subtle things that one needs to know to make sure that a research group is functioning properly. A lot of it has to do with making sure that the personalities mesh well. For a while, I tried to get all of my students to work together on One Big Project. We would have these big group meetings and write design docs but over time it became clear to me that it just wasn't working. It finally dawned on me that a couple of the students in my group (including one who had developed most of the core code that I wanted everyone else to rely on) were not that interested in working with other people -- they were far better off doing their own thing. I've also had students who really work fantastically in a team setting, sometimes to a fault. Those students are really good at doing service work and helping others, when they really should be more selfish and pushing their own agenda first. In general it's good to have a mix of personalities with different strengths in the group. If everyone is gunning to be head honcho, it isn't going to work.

Of course, most junior faculty go into the job with zero management training or skills. One's experience in grad school no doubt has a big influence on their approach to running a research group. My advisor was David Culler, who is known to be extremely hands-off with his students (though he can do this amazing Jedi mind trick -- to get his students to do his bidding -- that I have never quite mastered). I took after David, though I find that I am much happier hacking alongside the students, rather than only discussing things at the whiteboard. I also see lots of junior faculty who live in the lab with their students and have their hands all over their code, so there are many different paths to enlightenment.

All along I wished I had more management experience or training. Early on, I was given a copy of Robert Boice's Advice for New Faculty Members, and frankly found it to be fairly useless. It is unnecessarily cryptic: the very first section is entitled "Rationale for a Nihil Nimus (Moderate) Approach to Teaching" -- I am still not sure what the hell that means, but it certainly wasn't any help for someone starting up a big research lab.

It turns out that MIT Professional Education runs a short course on Leadership Skills for Engineering and Science Faculty. (I signed up for this a few years ago but they canceled the course due to low enrollment! I certainly hope to take it one day.) Another useful resource is the Harvard Business Review Paperback Series, which is a collection of (very short and readable) books on management topics, some of which are germane to science faculty running a lab. For example, the book on motivating people gets into the various ways of getting your "employees" (a.k.a. students) to be productive, and talks all about the pros and cons of the carrot versus the stick. Synopsis: If you can get inside the head of an unmotivated student and figure out what they want, you can motivate them to do anything. This must be the key behind Culler's Jedi mind trick.

First week at Google

2010-07-04T08:37:00.000-07:00

I started work at Google this week, and did orientation at the mothership in Mountain View. It was an awesome experience, and I had more fun than I have had in years. I certainly learned a hell of a lot. A bunch of "Nooglers" -- more than 100! -- were starting the same week, including Amin Vahdat, who is taking a sabbatical there as well. I've been asked a lot what I will be working on a Google. I can't provide details, but my job is a software engineer doing networking-related projects out of Google's Boston office. I won't be doing "research"; I'll be building and deploying real systems. I'm very excited.

Clearly, I haven't been there long enough to have any informed opinions on the place, but first impressions are important -- so here goes.

First, it should be no surprise that I'm blown away by the scale of the problems that Google is working on and the resources they bring to bear on those problems. Before last week, the largest number of machines I'd ever used at once was a couple of hundred; on my fourth day at Google I was running jobs on two orders of magnitude more. It is a humbling experience.
Having worked on and thought about "big systems" for so many years, being able to work on a real big system is an amazing experience. Doing an internship at Google should be mandatory for students who want to do research in this area.

The place is very young and energetic. There are few people over 40 wandering the halls. I was also impressed with the fraction of women engineers -- much higher than I was expecting. Everyone that I have met so far is incredibly smart, and the overall culture is focused on getting shit done, with a minimum of bureaucracy.

Orientation was a little chaotic. The very first presentation was how to use the videoconference system -- this did not seem like the right place to start. Of course, there is so much to learn that they have no choice but to throw you in the deep end of the pool and point you at a bunch of resources for getting up to speed on Google's massive infrastructure.

Google is famous for having a "bottom up" approach to engineering. Development is driven by short projects, typically with a few engineers with a timeframe of 3-12 months. Rather than a manager or VP handing down requirements, anyone can start a new project and seed it in their 20% time. If the project gains momentum it can be officially allocated engineering resources and generally the tech lead needs to recruit other engineers to work on it. (GMail started this way.) Inevitably, there is some degree of overlap and competition between projects, but this seems like a good thing since is rewards execution and follow-through.

Figuring out what the heck is going on can be pretty challenging. Fortunately Google has an internal search engine of every project, every document, every line of code within the company which helps tremendously. Internally, the corporate culture is very open and with few exceptions, every engineer has access to everything going on within the company.

I hope that I will be able to continue blogging about my Google experience -- their blog policy is pretty reasonable, though I won't be able to share technical details. But from now on I need to include the following disclaimer:

This is my personal blog. The views expressed here are mine alone and not those of my employer.

Cocktails for Computer Scientists

2010-06-19T17:11:00.000-07:00

One of the keys to success in academia is being able to make a good cocktail. Many a night I have slogged through a grant proposal or stack of recommendation letters for students with a Sazerac or Martinez in hand. One might think that regular imbibement of cocktails and the demands of faculty life are not exactly compatible, though I disagree. Moderation is important, of course; but not nearly as important as the requirement to stop working once you've finished off your third drink. It is a nice timeout mechanism.

Most of my friends know little about cocktails, or their idea of a cocktail is a boring old vodka martini or a margarita. (On the other hand, Michael Mitzenmacher is a fan of the Long Island Iced Tea, which essentially involves dumping whatever booze you have laying around into a glass.) Any time we have friends over I am in instant bartender mode and pushing new drink discoveries on them. So, I give to you, Matt's Guide to Classic Cocktails for Computer Scientists, or how to become a cocktail geek in four easy steps.

Step One: Use good ingredients. High-quality ingredients make all the difference in cocktail making, just as in cooking and any other endeavor. It was an absolute revelation the first time I tasted a proper añejo tequila -- sweet, smooth, not at all like that harsh stuff that you use to do shots. Those bottles of Jack Daniels and Bacardi you have left over from your college days? Chuck 'em. Stock up on a good, proper bar. At minimum, you'll need a nice bourbon or rye (Black Maple Hill or Hudson Valley); gin (Junipero or Plymouth); rum (Zaya or Ron Zacapa 23 Años); and sweet vermouth (Carpano Antica or Dolin). Vodka is totally unnecessary and best left for "fauxtinis". Other important ingredients for making real old-school cocktails include absinthe, maraschino liqueur, brandy, and the occasional exotic boondoggle like Batavia Arrack or Creme de Violette. If you are just getting started I recommend holding off on these extras until you need them.

Bitters are one of the most important components to a bar. My favorite, by far, are The Bitter Truth Jerry Thomas' Own Decanter Bitters. You should always have a bottle of Angostura on hand, and Peychaud's as well. Orange bitters are used in many classic drinks. Bitterman's Xocolatl Mole Bitters make a great conversation piece. I have six or seven kinds of bitters at my bar and my theory is you can never have too many.

Make yourself a batch of simple syrup: two parts sugar to one part water, boil in the microwave, put into a glass bottle in the fridge. You'll use it all the time. Spike it with a stick of cinnamon or a handful of whole black peppercorns and you can produce a masterpiece. Fresh lemons and limes should always be on hand. I tend to eschew olives and those gross nuclear-fallout colored "maraschino cherries."

Step Two: Get a decent cocktail book. Most of these are utter garbage. The worst are those that simply list a zillion cocktails in alphabetical order by name and don't tell you anything about the ingredients, history, or variations on the drink. The best classic cocktail book is Imbibe! by David Wondrich, which is a modern interpretation of Jerry Thomas' original cocktail guide from 1862 . (Reprints of the latter can now be found on Amazon!) Wondrich's writing is superb; the book is heavy on lore. If you want to make cocktails the proper, 19th century way, this is a good place to start. Online, CocktailDB is fairly comprehensive and makes it easy to search for recipes, but I find this approach overwhelming. Better are cocktail blogs such as Cocktail Virgin and spiritsandcocktails.com (among many others), which are more focused.

Step Three: Make some drinks. Now that you have your ingredients and your handy guidebook, what to make? I recommend starting with the very basics and go for an Old Fashioned: muddle bitters with a sugar cube and a little bit of water; add a couple of ounces pour of good bourbon or rye; stir with ice. That's it. This is not the Old Fashioned you will get at a typical bar (which loads the drink with a fruit salad).

Variations: Improved Rye (or Bourbon) cocktail: same as above, but add a dash of absinthe and maraschino liqueur, stir with ice and strain. A Manhattan made with proper bourbon and Camparo Antica vermouth is to die for. Substitute Fernet-Branca for the vermouth (and add a dash of simple syrup) and you have a Toronto Cocktail, probably my favorite drink these days. Flame an orange or lemon peel over the drink and you are gettin' really fancy.

A good gin martini -- with dry vermouth and orange bitters -- is an excellent thing. Use sweet vermouth instead (again, Camparo Antica) and sub Old Tom Gin instead of dry gin and you get a Martinez, the precursor to the martini, very 1880's. That one needs a lemon peel rubbed on the rim of the glass for full effect.

If you want to show off, make a Trinidad Sour -- a full ounce (!!) of Angostura bitters, orgeat (or sub simple syrup), lemon juice, a bit of rye. I guarantee you're not going to find that at Applebee's. This is for advanced cocktail drinkers only.

Step Four: Research. You can never go wrong by leaving it to the pros. In Boston, the best places for classic and modern craft cocktail making are Drink, Eastern Standard Kitchen, Green Street Grill, Craigie on Main, and Deep Ellum. Don't just order a drink that you already know -- talk to the bartender, get to know them, learn about the craft, get their recommendation. Drink encourages this by not having a cocktail menu; they expect customers to discuss what they like and don't like with the bartender. Usually when I go there I just tell them to make me something good, and they do.

Off to make up another Jamaican Ginger Cocktail #3...

Working for The Google

2010-06-17T07:18:00.000-07:00

I am about to take a one-year sabbatical from Harvard to join Google. If you haven't heard of them, Google is this little startup company with this really neat website that lets you search for just about anything on the Internet. It is very cool.

I'm going to be at the Google office here in Cambridge (since I can't move my family right now) but expect to work with folks at the Seattle and Mountain View offices. This way I can also keep tabs on my research group at Harvard and hopefully keep things moving along here. But largely I am going to be stepping back from my academic responsibilities -- I intend to fully dive into the Google job and immerse myself in that environment.

A lot of people have asked me what I'll be doing at Google. I can't say much, but I will hint that it has to do with using high-energy lasers to digitize real objects into a computer system called the MCP. I am sure nothing can possibly go wrong with this project.

Seriously, I only have a vague idea of what I will be working on, but the high order bit is that my job title is simply "software engineer." Essentially, I'll be writing code. Not leading a project or doing "research" or anything like that. To be honest, this makes me extremely happy -- I miss hacking on a daily basis and look forward to being a grunt in the trenches, so to speak. As far as I can tell, this is how most people come into Google, regardless of their level of experience. Craig Chambers, who left a tenured position at University of Washington to join Google, told me that he came in as a software engineer, hacking on someone else's project, and it wasn't until he had been at it for a while that he started defining his own projects and leading his own teams.

The way I think of it, being in academia is a lot like building toy boats and playing with them in your bathtub.

From http://www.flickr.com/photos/timothymorgan/522553650/

Of course, this has many advantages -- you get to completely define the experimental environment, don't need to worry about aspects of the boat design that don't interest you, and can explore radically new designs without much concern for legacy approaches or "making money." At the same time it can get pretty disconnected from reality, depending on how you approach it.

Whereas being at Google is like working on an aircraft carrier at sea.

From http://www.naval-technology.com/projects/cvn-21/cvn-211.html

In my case, my job will be to polish the portholes on the poop deck, and I won't have much influence on the overall design or direction of the ship -- but hey, I'll learn a lot in the process. There is also something very attractive about writing code that will actually be part of a running system and potentially used by millions of people every day. Of course, I'll have to get used to things like writing tests and doing code reviews, and working under a vast number of constraints that we tend to ignore in academic settings.

Keep in mind I've never really worked as a software engineer. In a lot of ways I feel totally unqualified to teach my students about writing good code or building reliable systems since I've never actually had to do it myself. (That's not quite true -- the research systems I build are subjected to rigorous stress testing, but certainly aren't mission-critical the way Google's code has to be.) A stint at Google will hopefully make me a better programmer, system designer, and researcher.

I'm not yet sure whether Google is going to let me continue blogging while I'm there -- hopefully they will let me blog about non-Google-related topics, but we'll see. Maybe I just need a pseudonym -- if you see a blog pop up called "Final and Centralized" you'll know it's me...

Editor in Chief

2010-06-14T13:43:00.000-07:00

In a bout of temporary insanity, I've agreed to serve as Editor-in-Chief of ACM Transactions on Sensor Networks, which is probably the top journal in the field. Feng Zhao, the Assistant Managing Director of Microsoft Research Asia, has been the EIC since the journal's inception some six (?) years ago, and has done a fantastic job building up the journal and putting together a fantastic editorial board. I've been on the editorial board for a while and overall the quality of the submissions is pretty high. It's an honor to be selected for this role and I hope to keep up the great work that Feng has started.

Systems people tend to eschew journals in favor of conference publications, but I still think that TOSN has an important role to play in the sensor nets community. We need a place to publish longer, more thorough papers than what you can typically cram into a 14-page conference submission. We need a place for papers of high quality that just won't make it into a decent conference -- retrospectives, position papers, surveys, and so forth.

Of course, a major problem with journal submissions (and TOSN in particular) is the very long review cycle. It's just not acceptable for papers to take a year (or more!) to get reviews back. Having served as an associate editor I know how hard it is to corral reviews from good people and get things done in a timely fashion. Of course, some AE's are better at cracking the whip than others. We also need more regular turnover of the editorial board membership to avoid burnout.

Coming into the job, I have three main ideas for how to improve TOSN's standing in the community. Of course I'm open to any and all suggestions beyond these:

1. Improving the connection between TOSN and sensor network conferences. I think it's important that we formalize a process whereby the best papers from SenSys, IPSN, and other venues are fast-tracked to TOSN. We should be more proactive about this and give authors a chance to use TOSN as a venue for publishing a "journalized" version of a good conference paper. This has been done informally for a while but I plan to make the process much more overt.

2. Encouraging submissions of survey papers, article versions of PhD dissertations, and retrospectives. We need more outreach to the community to make it clear that we want these kinds of submissions. Again, this is mainly about being more proactive about soliciting these papers and getting the word out.

3. Introducing special issues. Feng made the decision to avoid special issues in the early life of the journal, for the good reason that it was important to establish TOSN before branching out to specialized topics. Now that TOSN is well established, I think the time is right to have 1 or 2 special issues a year, especially on topics that may be difficult to publish in a conventional manner. One example would be a special issue on industry experiences with sensor networks. If you have suggestions for special issues (or better yet, would like to serve as editor for one!) please let me know.

Margo Seltzer is blogging

2010-06-09T18:18:00.000-07:00

My colleague Prof. Margo Seltzer has started blogging at http://mis-misinformation.blogspot.com/. She is joining the ranks of such distinguished Harvard Computer Science faculty who maintain blogs, including Michael Mitzenmacher, Harry Lewis, and Stuart Shieber. (Apparently, we have nothing better to do than stuff the interwebs with our crazed ramblings.)

Margo, welcome to the blogosphere. It's a lot like real life, only weirder.

How to get tenure at Harvard

2010-06-06T04:57:00.001-07:00

There is an old joke that says that at most universities, you have to write a book to get tenure, while at Harvard, they have to write a book about you. I am not sure who wrote that one, since I recently found out that I've been promoted to full professor with tenure. (Unlike most places, at Harvard, full professor is the only tenured rank. I've actually been an associate professor for three years now and the total clock is seven years.) So my time as a disgruntled junior faculty member is drawing to a close - on to the far more entertaining life as a (presumably) gruntled senior faculty member.

Harvard has a notorious reputation for not tenuring its own junior faculty. Indeed, some departments have not promoted from within for decades -- so long that they probably don't remember how to do it if they wanted to. In the math department, for example, junior faculty treat the job like an extended postdoc, with the goal of getting tenure somewhere else -- Yale or Columbia perhaps. You'd have to win the Fields Medal to get tenure in math at Harvard. Such departments treat the end of a junior faculty member's contract as an opportunity to scout out the best person in the world to fill the position, and typically the best person is 20 years more senior and at another school.

This is the classic Harvard model, but in recent years Harvard has started to use the term "tenure track" for the first time in its history. Since I joined Harvard in 2003, we have tenured six CS faculty from within, and turned down tenure to two people. The CS faculty here (and, more generally, the entire School of Engineering and Applied Sciences) are extremely supportive of junior faculty and we work hard to ensure that everyone has the best shot at tenure.

Unfortunately, this attitude is not pervasive, and often rubs against the antiquated culture found elsewhere in the university. For example, it was only recently that Harvard's request for tenure letters explicitly stated that candidate X from Harvard was actually under consideration for the job. The letters still request explicit comparisons against a set list of other faculty who are typically expected to be *full professors* at other schools, and respondents are asked to rank the candidates. To be promoted you need to be ranked first or second consistently across the letters. It is a very daunting process for a junior faculty member.

At many universities, tenure decisions are made at the department or school level, with the university essentially rubber-stamping those decisions. Not so here. The final step of the Harvard tenure process is the mysterious and fearsome ad hoc committee meeting, which is presided over by the President of the university, who has the final say. For this meeting, three senior faculty from other universities come and grill the internal "witnesses" that may support or oppose the case. I am pretty sure the meeting also involves a ritual with a human skull and a goblet of blood, but cannot confirm as of yet.

Now that I've passed the trial by fire, there is one last step. Harvard does not tenure anyone without a Harvard degree, and I've never been here as a student. So next fall, they will grant me an honorary Master's degree to clear that burden. I am not making this up.

From then on I hear it is just smooth sailing, lazy days with few responsibilities and just raking in the paychecks and use of the private parking space. Right? Right?

I'd like to thank all of the people who really made this happen. More than anything else, my tenure is a reflection on the hard work and vision of my amazing students and postdocs -- who took my wild-eyed whiteboard ramblings and turned them into reality. More often than not, though, the best ideas came from the students themselves. I have learned so much from them and have been extremely fortunate to have such an amazing group. I'd especially like to thank Margo Seltzer and Greg Morrisett for their tremendous effort in marshaling my case through the process. Thanks to Michael Mitzenmacher for the puff piece on his blog today. Finally, great thanks to all of my faculty colleagues for their encouragement and willingness to put up with my crap in our weekly lunch meetings.

(Once I've had a chance to digest it, I'll post a more personalized account of what it took to navigate Harvard's tenure process.)

The Secret Lives of Professors

2010-05-24T07:00:00.000-07:00

I came to Harvard 7 years ago with a fairly romantic notion of what it meant to be a professor -- I imagined unstructured days spent mentoring students over long cups of coffee, strolling through the verdant campus, writing code, pondering the infinite. I never really considered doing anything else. At Berkeley, the reigning belief was that the best and brightest students went on to be professors, and the rest went to industry -- and I wanted to be one of those elite. Now that I have students that harbor their own rosy dreams of academic life, I thought it would be useful to reflect on what being a professor is really like. It is certainly not for everybody. It remains to be seen if it is even for me.

To be sure, there are some great things about this job. To first approximation you are your own boss, and even when it comes to teaching you typically have a tremendous amount of freedom. It has often been said that being a prof is like running your own startup -- you have to hire the staff (the students), raise the money (grant proposals), and of course come up with the big ideas and execute on them. But you also have to do a lot of marketing (writing papers and giving talks), and sit on a gazillion stupid committees that eat up your time. This post is mostly for grad students who think they want to be profs one day. A few surprises and lessons from my time in the job...

Show me the money. The biggest surprise is how much time I have to spend getting funding for my research. Although it varies a lot, I guess that I spent about 40% of my time chasing after funding, either directly (writing grant proposals) or indirectly (visiting companies, giving talks, building relationships). It is a huge investment of time that does not always contribute directly to your research agenda -- just something you have to do to keep the wheels turning. To do systems research you need a lot of funding -- at my peak I've had 8 Ph.D. students, 2 postdocs, and a small army of undergrads all working in my group. Here at Harvard, I don't have any colleagues working directly in my area, so I haven't been able to spread the fundraising load around very much. (Though huge props to Rob and Gu for getting us that $10M for RoboBees!) These days, funding rates are abysmal: less than 10% for some NSF programs, and the decision on a proposal is often arbitrary. And personally, I stink at writing proposals. I've had around 25 NSF proposals declined and only about 6 funded. My batting average for papers is much, much better. So, I can't let any potential source of funding slip past me.

Must... work... harder. Another lesson is that a prof's job is never done. It's hard to ever call it a day and enjoy your "free time," since you can always be working on another paper, another proposal, sitting on another program committee, whatever. For years I would leave the office in the evening and sit down at my laptop to keep working as soon as I got home. I've heard a lot of advice on setting limits, but the biggest predictor of success as a junior faculty member is how much of your life you are willing to sacrifice. I have never worked harder than I have in the last 7 years. The sad thing is that so much of the work is for naught -- I can't count how many hours I've sunk into meetings with companies that led nowhere, or writing proposals that never got funded. The idea that you get tenure and sit back and relax is not quite accurate -- most of the tenured faculty I know here work even harder than I do, and they spend more of their time on stuff that has little to do with research.

Your time is not your own. Most of my days are spent in an endless string of meetings. I find almost no time to do any hacking anymore, which is sad considering this is why I became a computer scientist. When I do have some free time in my office it is often spent catching up on email, paper reviews, random paperwork that piles up when you're not looking. I have to delegate all the fun and interesting problems to my students. They don't know how good they have it!

Students are the coin of the realm. David Patterson once said this and I now know it to be true. The main reason to be an academic is not to crank out papers or to raise a ton of money but to train the next generation. I love working with students and this is absolutely the best part of my job. Getting in front of a classroom of 80 students and explaining how virtual memory works never ceases to be thrilling. I have tried to mentor my grad students, though in reality I have learned more from them than they will ever learn from me. My favorite thing is getting undergrads involved in research, which is how I got started on this path as a sophomore at Cornell, when Dan Huttenlocher took a chance on this long-haired crazy kid who skipped his class a lot. So I try to give back.

Of course, my approach to being a prof is probably not typical. I know faculty who spend a lot more time in the lab and a lot less time doing management than I do. So there are lots of ways to approach the job -- but it certainly was not what I expected when I came out of grad school.

Update 4/24/10: Thanks to Mike Belfrage for pointing me to this interview with Niklaus Wirth where he echoes some of the above sentiments.

Proposal: Improving the NSF Review Process

2010-05-14T06:15:00.000-07:00

Summary: Let's improve the NSF proposal review process by making it function more like conference program committees.

Intellectual merit: The core problem that this proposal addresses is the poor quality of many reviews submitted by NSF panelists. It is not uncommon for a proposal to be rejected with short, content-free reviews, offering little feedback to the authors. In many cases the scoring of a proposal is poorly justified, leaving the author mystified as to why they got such a low (or high) score. Recently, I had a proposal rejected where one of the reviews was essentially a single sentence in length. Not only does this not help the PI improve the work for later submission, but it leaves the impression that the review process is arbitrary.

(I'd like to emphasize that this is a problem that many NSF program managers have called attention to, but they are powerless as individuals to do much about it. So I believe the fault rests with the research community, not with the NSF PMs.)

A key problem with NSF panels is that there is no community standard for what constitutes a good (or even acceptable) proposal review. I am a strong advocate of the approach used by the systems community, where paper submissions are given extremely detailed reviews with constructive feedback. Given that we spend so much effort reviewing papers, couldn't we also give the same effort to NSF proposals, which arguably are more important than a single paper?

It is my impression that NSF program managers also have a hard time pulling panels together, mainly because people are so busy, and don't have the time to travel to DC. Yet many of the potential panelists freely serve on conference program committees with much higher reviewing loads and an expectation of much more detailed reviews. (A typical panelist will review 8-12 proposals, whereas a competitive conference will require TPC members to review 2-3x as many papers.) Why? One reason, perhaps, is that program committees are recognized for their work, and serving on a TPC is an important indication of one's stature in the research community.

These two issues are related. Since serving on an NSF panel is seen as "paying your dues," rather than an activity you take pride in, there is little incentive to write good reviews. However, if you write a bunch of crappy reviews for a TPC, you can earn a reputation as someone who doesn't take the process seriously might not get invited back in the future. So the public recognition of the TPC and the quality of the reviews go hand in hand.

My proposal: Let's have NSF panels emulate the conference program committee model. First, we should recognize panelists publicly for their work. Being on the "NSF NeTS 2010" panel should be as prestigious as serving on SIGCOMM or SenSys. The NSF should create a web page for each years' competition where they list the panelists and list the proposals funded through that competition (the latter information is available, but a little hard to dig up). So the community can take pride in the effort and see the outcome of the process more directly.

Second, establish expectations for high-quality proposal reviews. If you are a bad panelist, you won't get invited back in the future, so you won't gain the recognition of being asked to serve. Panelists will be chosen from among the best people in the field, where "best" is defined both by research contribution and service.

Third, hold panels somewhere other than Washington DC. Since I live in Boston, it's easy for me to get down there, but for people on the West Coast it is much harder. If panels are run in different locations around the country, the travel burden can be spread around more evenly.

I will be the first to admit that the conference program committee model is not perfect -- see my related posts here and here for thoughts on that. But in my experience it is better than the (typical) NSF panel.

Of course, NSF's conflict-of-interest guidelines will have to be tweaked. Currently, you can't serve on an NSF panel for a program for which you have submitted a proposal. (The upshot is that panels tend to consist of people who didn't get their act together to submit a proposal, which may not be the best group of scholars to evaluate the work.) Recently NSF has separated larger programs into multiple panels and simply ensured that someone can't serve on the specific panel for which their own proposal is under consideration.

Broader impacts: Improving the proposal review process and providing more constructive feedback to PIs will result in better science.

In the comments, please indicate your rating for this proposal:
   Excellent
   Very Good
   Good
   Fair
   Poor

Geoff Challen and IDEA

2010-05-13T17:29:00.000-07:00

First of all, I'm very pleased to report that my student Geoffrey Werner Challen (formerly known as Geoffrey Werner-Allen, for reasons explained on his own web page) just defended his thesis! Geoff was the lead student on our work on sensor nets for volcano monitoring and developed the Harvard MoteLab testbed (used by dozens of research groups around the world). His work on Lance and IDEA defined new paradigms for resource management in sensor networks. He will be joining the faculty at the University of Buffalo next year and I am very proud of him. Congrats!

Second, I just posted the final version of our MobiSys'10 paper on Integrated Distributed Energy Awareness for Wireless Sensor Networks. This paper deals with the problem of configuring individual nodes in a sensor net to achieve some network-wide energy objective, such as targeting a given network lifetime. Each node generally has many local parameters that can impact overall energy consumption, such as the choice of parent in a routing tree or the MAC protocol duty cycle. We observe that nodes performing a greedy, local decision will often get it wrong; for example, a node deciding to forward packets through a parent with limited energy can rapidly drain the parent's supply. When energy harvesting (such as solar charging) is employed, it becomes very difficult to tune each node to achieve a network-wide objective.

The idea behind IDEA (sorry!) is to enable individual sensor nodes to make decentralized decisions about how to configure their own state. Nodes share information on their own battery level, charging rate, and energy load with other nodes in the network, using an efficient neighborhood broadcast scheme (akin to Abstract Regions). This information coupled with a utility function representing the intrinsic value of each possible state (such as a choice of parent node) allows nodes to make informed decisions that account for the non-local energy impact. In the paper we show that IDEA can significantly improve energy load distribution and network longevity with low overhead.

The PC Meeting Protocol

2010-05-05T19:18:00.000-07:00

I am always surprised at how chaotic program committee meetings tend to be. Although most people have served on several PCs, it seems that a lot of the same procedural questions and issues come up each time, and it would be helpful to establish a common protocol for the community to maintain order. Having just gone through the SIGCOMM TPC meeting (with a whopping 50 PC members - it was like being a delegate at the UN) I started thinking about some of the things we could possibly standardize to make the process run more smoothly. (By the way, Geoff and KK did an awesome job running the meeting - the problems outlined below are common to *all* PCs I have been on!) Michael Mitzenmacher not-live-blogged about this meeting here.

The first is laying down the ground rules. Program chairs tend to assume that PC members know basic things like not to leak information during the PC meeting (emailing your students or colleagues when the paper is being discussed), not to express an opinion on papers you didn't review, and not to compare the paper to the version that was rejected last year. This stuff seems obvious but it's amazing how often people forget.

The order in which papers are discussed is another important decision. In this case there is no one-size-fits-all model. In my opinion, the best model is to group papers by broad topic areas, and within each area discuss the best and worst papers in the group first, followed by those in the middle of the group. This helps to calibrate things and keeps the discussion focused on a set of papers with some commonality. The worst model, I think, is to discuss papers by order of submission number (which is effectively random), since people don't get calibrated that way.

Keeping the discussion to a time limit is very important. Many PC members think that it's their job to hold forth about every minute detail of the paper during the meeting. Once, there was a case where the lead discussant went on for about 6 minutes about a paper, and when finally cut off and asked what he thought the decision should be, said "I don't care!" PC members need to remember that the audience for this discussion is the chairs and the other reviewers (the other PC members are usually reading email or otherwise ignoring discussion of papers they didn't read). So keep it short and sweet, and respect everyone's time.

Dealing with conflicts is always a huge pain. It is disruptive to ask people to leave the room and then call them back in later. It would be awesome if HotCRP could automate this (coming up with a paper discussion schedule to minimize the number of times someone has to leave the room). I'd like an iPhone app that automatically buzzes people when they should leave and come back into the room -- a lot of time is wasted in PC meetings with these context switches.

It has to be recognized that reaching consensus is not always possible. PC members have to accept that they will win some and lose some. I dislike it when the discussion comes down to a vote amongst the reviewers, since that is rarely a good way to decide these things, but at least it is quick and painless.

The last issue is the role of shepherding. This should be explained clearly by the PC chairs at the start of the meeting. Personally, I am in favor of shepherding every paper for a major conference, with the goal being to ensure that the final paper really satisfies the reviewers' main comments and the writing is cleaned up (as it often needs to be). In general, shepherding should not be used to get the authors to run new experiments or change something technical in the design -- it should be limited to changes in wording or positioning of the work. This question comes up at every PC meeting I've been on and setting expectations early would make things easier.

Check out my related post on the psychology of program committees for a behind-the-scenes look at what happens behind the closed doors.

Why I am a Computer Scientist

2010-04-29T18:47:00.000-07:00

As a counterpoint to my last, somewhat pessimistic post on the future of CS, I thought I'd post something more upbeat today.

Ed Lazowska from UW gave the colloquium at Harvard this week on Computer Science: Past, Present, and Future. This is a talk he has no doubt given many times at many places, though it was the first I had heard it, and it was fantastic. More than anything else, his talk reminded me of why I am a computer scientist, and of how many great problems we have to work on in this field. I can't imagine wanting to do anything else.

Ed's talk started off reviewing what computer science has accomplished in the last 40 years, since the ARPAnet came online in 1969. This New York Times story from 2009 reported on the "Top 20 inventions of the last 30 years" and nearly all of them are derived from computer science in one fashion or another -- the Internet, PC, mobile phones, and email top the list. Although there's no surprise here at all, it was a great reminder of how much impact computing has had on nearly all aspects of the modern world.

This is why I am a computer scientist: because I want to change the world. Computing is the engine that is driving innovation in every field of humanity, and working on computer science puts you at the center of it. Of course, it's easy to forget that on days when I am beating my head against some obscure Objective C type mismatch bug, so it's nice to get a reminder of the bigger purpose.

This is why I am a computer scientist today, but it's not how I got started. My first experience with a computer was sitting down at an Apple II+ (I am dating myself) when I was 8 years old. I remember it clearly: The teacher told me to type my name at the prompt and I got:

]MATT

?SYNTAX ERROR

This was probably not the most rewarding first experience with a computer, but it taught me that this box spoke a different language, and I wanted to learn how to communicate with it. It was not long before I was writing BASIC programs to do lo-res graphics. While most kids in the class just manually translated their static image from a sheet of graph paper into a series of PLOT, HLIN, and VLIN commands, I was writing animated scenes (one was the pyramids with an animated sunset behind them) and even a sword-fighting game with a simple AI algorithm so you could play against the computer (keep in mind this was 3rd grade). I was totally hooked.

At that age, computers represented this amazing magical box that you could control and make it do almost anything. The games I was writing back then rivaled what I could buy in the store, but for me it was much more about writing the code than playing the game.

Now that I have a son, I wonder what his experience with computing will be like, and whether he'll be as turned on as I was by the whole mystery of the thing. Perhaps he will just treat the computer like any other appliance, like the dishwasher or telephone -- not something to be taken apart, programmed, explored. One thing we already do is play with the iPad, and even at 9 months old he loves to touch the screen and manipulate objects on it -- there are a couple of good iPad and iPhone programs for babies and the touch screen interface has tremendous potential there. But will he want to program? And how will he even do it? BASIC and LOGO were great back in the day as you could dive right in. I'm pretty sure I don't want to throw Visual Studio at him -- hopefully there are still good kid-friendly programming environments out there. And of course, the programs he'll be able to write won't be able to hold a candle to whatever the latest 3D immersive video game system he'll be playing then, so it's hard to say whether he'll appreciate the potential for doing it himself.

I am convinced that giving kids this very direct and rewarding experience with computing is important, though. If we turn computers in to just another kind of media consumption device (which most already are) then we'll lose that opportunity to hook the next generation.

Will DARPA save computer science?

2010-04-26T17:45:00.000-07:00

I've been doing a lot of thinking lately about the role of academic computer science research vis-à-vis the state of the art in industry. When I was in grad school at Berkeley, we were building "big systems" -- the 200+ node cluster that I did all my research on was on the rough order of the size of sites like Inktomi and Google at the time. Since then, industry has scaled up by orders of magnitude, leaving academics in the dust. These days it's not clear that it makes sense for a university research group to work on systems problems that try to approximate industry: not only do we not have the resources, we simply have no idea what the real problems are at that scale. My friend and colleague Steve Gribble told me that after doing a sabbatical at Google, he decided that there's no way a university group could compete with what they're doing.

So what should academics be spending their time on? Too many academic systems researchers, I think, are doing "industry research in the small", with project horizons on the order of 2-3 years, not things that are radical departures from the state of the art. Is this the right approach? In a large department, it might be possible to approximate industry; David Patterson's PARLab at Berkeley is an example. This takes a tremendous amount of industry funding, though, and it's not scalable in the long run -- and there are not many people like Patterson who can pull that off. So what are the rest of us to do?

Ideally, academics should be pushing the envelope of computer science well beyond where industry is looking today. My research at Harvard has focused on radically new computing platforms -- mostly wireless sensor networks -- and our RoboBees project is pretty out there too. The downside is that industry isn't as interested in these problems, so it's harder to get funding from industry when you're not working on problems that are on their critical path. The other problem is that working on sci-fi it's more difficult to have impact on the real world, unless you're willing to wait for many years.

DARPA could be our savior. When I started my faculty job in 2003 I was told that getting money from DARPA would be no problem, but during the Tether years it mostly dried up. As a result nearly all of my research at Harvard has been funded by relatively small NSF grants plus various industry gifts. I am very hopeful that with Peter Lee heading up the new TCTO office that we'll see bolder initiatives from DARPA to bring back the good old days.

In the meantime, I guess I could find some bugs to squash in the Linux kernel...

Should Harvard's Intro CS class do away with grades?

2010-04-19T06:27:00.000-07:00

We've been having some discussion amongst the CS faculty over the last few weeks about whether CS50, the intro computer science class at Harvard, should do away with letter grades and instead switch to SAT/UNSAT. Recently, the Harvard Crimson reported that CS50 is going to do away with letter grades, but this is not true -- the issue is still being debated by the faculty, and has yet to have any formal approval. (I don't know what led the Crimson to report it as though it were a fait accompli.) Given that my opinion seems to differ from most of the CS faculty, I thought I'd put my two cents forth here. Of course, this only represents my own thoughts on the matter, not the CS faculty as a whole (so if you are a Crimson reporter, don't go around reporting this as the final word).

For the record, I used to teach CS61, one of the two follow-on courses to CS50, so the output of CS50 directly feed into my course, and I have a vested interest in the intro course maintaining a certain standard. (My colleague Steve Chong is taking over CS61 next fall.)

David Malan, who teaches CS50, has done a fantastic job at increasing enrollments since taking it over 3 years ago -- I believe the enrollment has more than doubled (possibly tripled) under his watch. CS50 is going great guns and attracting students from all over the University, including many students who would have otherwise never taken a CS class. (Unlike a lot of places, CS is not required for Harvard undergrads as a whole, although it is required for a number of majors, including engineering, applied math, etc.) The course culminates in the CS50 fair where students show off their final projects. It is a great course and David has done amazing things to raise the profile of CS at Harvard.

So, right off the bat, I question the need to change the grading option for CS50, which potentially creates more problems than it solves.

David says that he has been wanting to get rid of letter grades entirely in CS50 for some time, and this year decided to raise the issue for discussion. David's claim is that there are still a lot of Harvard students who are intimidated by CS50 (despite the massive trend in increased enrollment) and that doing away with grades will make those students feel more comfortable trying out CS. While this may be true, I am not sure that we should be designing our foundational intro courses for those students. Harvard already has a "CS for non-majors" course, CS1, to fill that need.

A number of faculty believe that more women will be attracted to CS if they can take the course without worrying about their grade. I can't say whether that is true, but I find it somewhat implausible that what is standing between women and CS is just letter grades.

Even if you believe that doing away with letter grades will increase enrollment, consider the possible downsides. The first is that students who complete CS50 with a "SAT" grade won't have a good idea of their readiness to take more advanced CS courses. Grades do have informational value, and eliminating them makes it much harder to plan one's future course of study. The second is that the course will be less attractive to students who intend to major in CS or engineering, or who are simply used to getting all A's (as many incoming freshmen at Harvard are), so this change could actually decrease enrollment at the top end of the course. It would be fine if the hardcore students could just skip CS50 and take one of the later courses instead, but those courses are not currently designed to supplant CS50 for the better-prepared students. Also, skipping CS50 means missing out on the community experience that I think is important for student cohesion.

The third, and most severe, problem with this proposal is that it will make CS50 no longer suitable for the ABET-accredited engineering degree (which requires letter-graded courses) and the University-wide General Education requirement. So by trying to cast a wider net with CS50, the course no longer satisfies important requirements for certain students, so this approach seems self-defeating.

The compromise proposal currently on the table is to offer CS50 in two flavors: a letter-graded version (call it CS50g) and a SAT/UNSAT version (call it CS50su). It would be the same exact course, just with different grading options. This way students who need the letter grade can take CS50g, and everyone else can take CS50su. It seems clear that this will backfire in several ways. First, many students who take CS50su will later realize they needed a letter grade to satisfy a requirement down the line, but it will be too late. Second, it will create a class distinction between the CS50g and CS50su students. Students taking the course for a letter grade will demand much more detailed feedback on their assignments and more rigor in the grading process (since they will be competing for A's) whereas the course staff, used to dealing with mostly SAT/UNSAT students, will not feel the need to make such fine distinctions.

I have not looked into what other universities do and whether there is any agreement that doing away with letter grades for an intro CS course is a good idea. I know that MIT grades all freshmen pass/fail to alleviate stress associated with grades, which might make sense if done across the board, but it is unclear that changing one class is the right way to do this.

Personally, I'd rather not mess with CS50 right now. It's doing great things and I am concerned that doing away with letter grades would do more harm than good. Of course, many of my colleagues here at Harvard disagree with me, and they are encouraged to weigh in on the comments section below.

Update 4/20/10: The Crimson published a follow-on article this morning discussing the CS50 controversy.

Update #2 4/26/10: Given the many concerns raised, there was simply not enough time for the CS50 SAT/UNSAT proposal to be discussed by the faculty before the deadline for changes to the course catalog for next fall. As a result the proposal has been tabled for now; I expect it will be revisited next year.

HotOS XIII Call for Papers

2010-04-07T16:43:00.000-07:00

I am pleased to announce that the call for papers for the Thirteenth Workshop on Hot Topics in Operating Systems (aka HotOS XIII) has been announced. I am fortunate to serve as the program chair and we have put together a fantastic program committee for the workshop, which will be held from May 8-10, 2011 in Napa Valley (think good wine and fantastic food... No promises yet on whether Thomas Keller will be doing the catering). While it's a long ways off, it never hurts to start thinking ahead about what you want to submit! The paper deadline is January 15, 2011.

A few words about HotOS and my own philosophy behind the workshop. HotOS has been the flagship venue for bold new ideas in the systems community over the years. It is often the first place that we hear about new projects and exciting ideas, and it's also a place for grad students to float their crazy thesis plans before submitting full papers to places like SOSP and OSDI. Just to be clear on the format: HotOS submissions are five-page position papers, which are not to be confused with miniature versions of full conference papers (i.e., with half-baked ideas and none of the graphs). The best submissions to HotOS are those that challenge widely-held assumptions, make a clever or unorthodox argument, and get people thinking and talking. In my opinion, the ideal HotOS paper has a strongly-worded idea that gets people working on new problems, without concern for how practical the idea is in the short term. Graphs and early prototype results are actually a negative here -- if your idea has progressed to that point, it is probably not a good candidate for HotOS.

One common complaint about HotOS is that it is sometimes heavily loaded with "SOSP preprints," and indeed, there is a pretty high conversion ratio from HotOS paper to SOSP paper in the same year. Not everyone thinks this is a bad thing and I agree this does serve a certain purpose. As program chair I plan to exert my influence to keep things focused on the bleeding edge, but I also respect that different members of the TPC will have their own taste for different styles of papers.

Beyond the technical content, HotOS serves an important role of building ties between members of our community, and especially to help grad students get a chance to engage, mano a mano, with more senior folks in the field. Mentorship is an incredibly important part of the workshop. I'll never forget my first HotOS and the chance to bat around ideas with such luminaries as Jay Lepreau. There is also a fair bit of, shall we say, enology involved at HotOS, which certainly makes things more interesting (distributed hash tables have never been more fascinating!). There is a huge difference between meeting folks at a 400+ person conference versus a small workshop where people are more likely to let their guard down.

I want to build upon this tradition and make HotOS more inclusive to new folks and less-represented research groups. I am going to be on the lookout for unusual, bold, oddball papers from outside of the "conventional" systems community, with the hope of shaking things up a bit, but it's safe to say that there will be plenty of familiar faces as well. If you have a nutty idea, by all means submit it. I'm going to try to tease apart novelty from "solid work" in the reviewing process, and take a range of papers that have different kinds of strengths. We'll also be inviting some folks who don't submit papers at all just to be sure we have a good mix of disciplines and groups represented.

If you have thoughts or suggestions for HotOS, please feel free to drop me a line or post comments here. And I look forward to your submissions!

Reviewing papers on an iPad

2010-04-07T06:01:00.000-07:00

Following up on my recent post on Mac tools for profs, I wanted to share some early thoughts on the use of the iPad for reading and reviewing papers. (This seems to be 60% of my job these days, and it was the main reason I got an iPad in the first place.) For the last year or so I've gone paperless with paper reviews: I read PDFs on the laptop screen and have a separate text editor window open to type up my review. So the iPad seemed like the perfect way to carry the proverbial stack of papers around with me and write up reviews anytime, anywhere.

I've been testing a bunch of iPad apps for paper reading and annotation. Verdict: The software for this is still immature, but it's clear that the potential is there. In a few months I hope this will be a lot more straightforward.

Good news first: Reading PDFs on the iPad screen is fantastic. Although the screen is a little smaller than a 8.5x11" or A4 page, you can still read the text quite clearly and every reading app lets you zoom in (so you can, for example, eliminate the margins on the page or zoom in on a single column). The multitouch interface makes this easy to do and there's something quite visceral about panning and zooming around a paper with your fingers. (On my wish list: A PDF reading app that literally allows you to "poke holes" in papers as you read them, or crumple them up with a multitouch gesture.)

The PDF readers can handle very large documents and graphics and images are rendered just fine. I recently reviewed a 170+ page grant proposal on the iPad and had no problems with it at all.

Now for the bad news: First, annotating PDF files or taking notes is still somewhat crude, depending on the app. Some of the best reader apps, like GoodReader, don't support annotations at all. (I wouldn't be surprised if they added this feature sometime.) The best annotation support I've seen is iAnnotate, which lets you do pretty much anything (notes, highlights, scribbles, bookmarks), and the annotations can be read by any standard PDF reader. However, the user interface is a bit clunky and syncing PDF files is somewhat of a pain (see below). My favorite app so far, Papers, only supports plain text notes that are kept separate from the PDF file, so you can't just email a marked up paper to someone.

The other piece of bad news: Getting PDFs onto the iPad, or getting your notes off, can be a pain. Unfortunately, the iPad OS does not support a common filesystem across apps, so each app needs to have its own way to sync files to the device. This means that if someone emails you a PDF file, you can't just save it and load it up into one of the apps mentioned above.

The best sync support by far is GoodReader, which can pull from darn near anything: the Web, an FTP or WebDAV server, DropBox, Google Docs, even email attachments (which you configure separately from the Mail app). Unfortunately, there is no way to annotate PDFs and hence no way to get documents off the iPad. I hope they change this since they've clearly done a great job with the connectivity options.

iAnnotate and Papers have their own custom sync methods that require that you run an app on your Mac (or PC in the case of iAnnotate) and do the sync over the wireless LAN. They don't let you sync directly through iTunes, though again, this would be an obvious addition in the future. I already use Papers on the Mac to track my ever-growing list of papers to read, so this was the obvious choice for me. The UI needs a little tweaking: when reading in landscape mode, you have the Library pane taking up part of the display and there's no way to hide it. This would be easy to fix.

What is lacking is an all-in-one solution that lets you sync from anywhere, maintain metadata, and annotate. A Frankenstein of GoodReader, iAnnotate, and Papers is what I really want. Even better would be direct integration with HotCRP and the ability to edit and upload the review form directly. Hmm, maybe I'll have to write this app myself...

The Psychology of Program Committees

2010-03-30T06:14:00.000-07:00

One thing that I frequently tell my grad students is that your chances of getting a paper accepted to a conference depend as much on who the reviewers are, and what kind of mood they are in, than the content of the paper itself. OK, leaving aside those obviously brilliant papers that my group is known to churn out, the paper content does matter -- but on most program committees there is a large pile of "pretty good" papers that all have a roughly equal chance of getting accepted. In these cases it often comes down to the collective mindset of the reviewers during the PC meeting.

There are many subtle psychological effects that influence the disposition towards a given paper. The first has to do with the timing of the paper discussion. At the beginning of a PC meeting, everyone is amped up on caffeine and uncalibrated with the respect to the overall quality of the papers being discussed. Your chances of getting a paper accepted when it is discussed early on in the meeting can vary widely. Most PC meetings discuss the top-ranked papers first, but after a string of (say) five or so papers accepted, people start thinking that it's time to reject a paper, so the sixth paper tends to be a scapegoat to release the pressure and make everyone feel better that the conference is still "selective." Fortunately, most PC chairs recognize this effect and switch gears to discussing some of the lower-ranked papers next, so the committee sees both ends of the spectrum. I've been in PC meetings where the top ranked paper with four strong accepts and a weak accept is tabled for further discussion, just because the committee is thirsty for blood.

Late in the afternoon, everyone is exhausted, cranky, and the power structure of the PC meeting has started to play itself out -- who's willing to throw themselves on the tracks to accept or reject a paper; who's willing to defer to more senior members of the committee; etc. Here we start to see some interesting personality traits emerge that can save or sink a paper:

"I didn't read this paper, but I know the work." This drives me nuts -- someone who was not even a reviewer casting doubt on (or supporting) the paper being discussed because they know the people involved or had some discussion with them at a conference a few months ago. This should be flatly disallowed by the program chairs, but I've seen it happen. A variant on this is...

"I didn't read this paper, but I saw their original failed submission last year." Again, this should be disallowed by PC chairs -- whether a paper was any good last year should have no bearing on the discussion of the present submission.

"I'm not an expert in this area, but I don't think there's anything novel here." Too many times a reviewer who is simply not qualified to review a paper is unwilling to defer to more expert members of the committee. Someone who doesn't know the related work that well might infuse the discussion with a vague sense of unease that taints the rest of the reviewers and makes it harder for someone to champion the paper for acceptance.

"I know way too much about this area and they should have used a value of 1.3 instead of 1.4 for the alpha parameter on page 7." Often, when a paper is too close to a reviewer's area, they tend to nickle-and-dime the paper for small problems that chip away at its credibility. Sometimes this is a poorly disguised attempt at tamping down the competition. These kinds of reviewers often miss the forest for the trees, where a paper has some good ideas but needs some rough edges sanded off, as all papers do.

"I'm a new faculty member and want to prove how smart I am by rejecting most of the papers in my pile." When you are new to program committees there is a real temptation to exercise your new power by rejecting papers left and right, which clearly establishes your intellectual dominance over the poor authors who are at your mercy. Most new faculty fall into this trap, and I've certainly been in this situation before.

"I'm a senior, well-respected faculty member and like to compare all of the papers in my pile to things published in the 1960s." The "there's nothing new here" argument sometimes comes up when you have a senior, somewhat jaded PC member who thinks that all of the good ideas were published back in the good old days when men were men and the women programmed in octal. It's inevitable that good ideas will come up time and time again, and I actually think there is value in reevaluating previously-published ideas in the context of new technology capabilities and application demands. Perspective is a great thing but sometimes you can have too much perspective.

The final point is that it is easy to argue to reject a paper; much harder to argue to accept a paper over other reviewers' objections. If a reviewer is not really sure about the novelty or importance of a paper's contributions, they often defer to the most negative reviewer, since nobody likes looking like an idiot in a PC meeting. Standing up and championing a paper takes a lot of guts, and by doing so you are taking responsibility for any faults in the paper that might arise later (if it turns out, say, that the exact same idea was published before, or there is a flaw in the experimental design). I think it's important that every member of a program committee commit themselves to championing one or two papers during the meeting, even if they aren't so sure about them -- otherwise the process can get to be too negative. One way to view your role as a PC member is to identify that beautiful piece of work that would otherwise have been overlooked due to superficial problems that turn off the reviewers.

So, next time you get a paper rejected, just remember that it's probably because the reviewers were in a bad mood because they hadn't served the afternoon coffee yet.

The App Store is evil. And I love it.

2010-03-28T17:58:00.000-07:00

Apple's App Store is the perhaps the most brilliant innovation in software distribution ever. In case you've been living in a cave, the App Store lets iPhone and iPod Touch (and soon, iPad) users download and install apps directly on their device. This has absolutely revolutionized the way that software applications are marketed and sold. With a single tap you can download an app and the price is automatically billed to your credit card. The best part is that Apple gets to keep 30% of the app price. So, every sale of the $900 iRa Pro app nets Apple $270. They must be raking it in!

Before the App Store, installing apps on mobile devices was a huge pain. My old Windows CE PDA required that you download a ZIP file to your Windows machine (a deal-killer right there), unpack it, run a wizard, physically tether the PDA to the PC, go through several steps to complete the installation, and usually reboot a couple of times for good measure. Any time the PDA's battery ran out I had to create a new device profile in Windows and re-install the apps by hand. It was totally broken and it is unsurprising that the application market did not exactly take off on these platforms. On the App Store, a purchase is just a tap away. I'll admit to have bought quite a few iPhone apps on a whim, maybe because I was about to board a flight and wanted a new game to try out. Dropping 99 cents on a new app does not seem like a big deal at the time, but if I were to add up all my app purchases in the last year, the total is no doubt in the triple digits.

Of course, the App Store is also blatantly, totally evil. It gives Apple a monopoly on the software distribution channel. Now, I'm all for quality control -- it's nice that Apple is trying to screen apps to ensure some modicum of sanity, and perhaps to screen out trojans and such -- but this is going to have a profound effect on how developers and users interact in the future. Essentially, the App Store means that the person owning the device has no control over what software can and can't be installed on that device. This is a huge philosophical shift from our current model, in which the hardware manufacturer, OS developer, and application developers were all separate entities. Apple is doing a great job at consolidating power for its platforms, which of course includes not just apps but also music, books, video, and other media.

I'm a big Apple fan boy so I find myself somewhat unnerved by these developments. When it was limited to these little cheesy mobile devices like the iPhone, the totalitarian App Store model did not seem like such a problem. Now it's being expanded for the iPad, and it would not surprise me to see an App Store-like model for conventional desktops and laptops in the future. This has dire implications for freedom and openness, which I think is important for the future of technology. As much as I like Apple's products, I think it's dangerous to let one company decide what we can and can't install on a device. In five years when everyone is using iPads instead of laptops, how are Computer Science students supposed to tinker and learn to program on their own when they have to go through Steve Jobs' army of goons before they can even run their own code?

Of course, I'm eagerly awaiting my iPad delivery this weekend. And I can't wait to drop $5 for the iPad version of Flight Control -- it's going to be awesome!

Mac tools for profs

2010-03-25T11:52:00.001-07:00

Macs seem to be insanely popular amongst CS faculty. Most conferences and faculty meetings I go to are dominated by Mac users. No big surprises, since (a) Macs work, and (b) they're sexy. I switched from Linux to Mac a couple of years ago after I got tired of editing three configuration files and rebooting to join a wireless LAN. That worked when I was a grad student, but now I'm too busy for that kind of crap.

I wanted to share some links to good Mac specific tools that I've found to be very useful in my job. If you have other suggestions, please share them as comments!

OmniGraffle is a great figure drawing program and produces very professional results. It's also easy as hell since it can do most of the layout for you, making sure that the boxes and arrows all line up correctly. The PDF output looks very slick and I've been using it for most of the figures in my papers; see Figure 1 in our SenSys'08 paper on Pixie for an example. Be sure to get the educational pricing.

Papers is one of my favorite Mac apps. It's like iPhoto for PDF files -- it will keep track of all of your papers, index them by title, author, keyword, etc. I use this program to keep track of my ever-growing reading list (rather than printing out a bunch of papers and letting them collect dust on my desk.) It also has a pretty slick interface for matching metadata about a paper (e.g., to pull in full citation information from the ACM Digital Library or Google Scholar) and for exporting to BibTeX and other formats. You can take notes and there's even an iPhone app (and soon, an iPad app) to let you read and take notes on papers on the go. (Yes, reading a two-column paper on the iPhone screen works -- if you zoom in on a single column and go widescreen, the text is the same size as on a printed page.) The only downside is that you have to manually synchronize the Papers library across multiple machines, easily accomplished with Unison, but I wish that were simpler.

BibDesk is a BibTeX library organizer. To import a new BibTeX entry, just copy it to your clipboard and paste it in BibTeX -- everything appears in the correct fields. You can also drag and drop BibTeX entries between files. This is infinitely easier than editing BibTeX files by hand and keeps the formatting right.

OmmWriter is a fantastic little app that is a essentially a Zen text editor -- it clears your entire screen and shows you only the text that you are writing. This is a great way of minimizing distractions and focusing while you write. WriteRoom is similar but I find OmmWriter's interface more appealing.

Finally, Caffeine is a little app I could not do without -- it puts an icon in your menu bar that, when clicked, disables your screensaver. This is very important when giving talks and avoids the embarrassing moment when your screen saver kicks in mid-presentation.

Who pays for conference reviews?

2010-03-08T07:23:00.000-08:00

Why not make authors pay to submit papers to conferences?

Serving on a program committee takes a tremendous amount of time. So, one of the frequent complaints that TPC members make is when authors submit half-baked, clearly below-threshold papers a conference just to get some reviews back on their work. Personally, I feel little responsibility to write detailed reviews on papers that are clearly in the "Hail Mary" category, but I still have to read them, and that takes time. Not to mention the long-term psychological damage incurred by having to read a slew of crappy papers one after the other... I'm still in therapy after IPSN 2007 :-)

The problem is that submitting a paper to a conference is free: all it takes is a few clicks of the mouse to upload your PDF file. (Of course, I'm not accounting for the cost of doing the research and writing the paper itself.)

Let's estimate the costs associated with serving on a program committee and reviewing a stack of papers. I spend about an hour reading and writing a review for each paper that I am assigned. A highly competitive conference will assign 25 papers (or so) across one or more reviewing rounds to each TPC member, equating to roughly 25 hours of my time. At my current salary, that is worth around $1900 (give or take). Then there is the PC meeting itself. This will typically involve two days' worth of work plus travel -- let's estimate 3 full days of labor, plus airfare and hotel, adding up to another $2500.

So, with a program committee of 18 people, that works out to around $79,000 to review something like 150 paper submissions. In other words, to recoup its costs, the conference should charge authors $500 just to submit a paper. This seems to make a lot of sense.

Of course, imposing this kind of a fee would no doubt drastically reduce the number of papers that are submitted. But this seems like a good thing: it would reduce the workload for TPC members, allow conferences to operate with smaller, more focused program committees, and vastly improve the quality of the submitted papers. It would potentially also improve the quality of the reviews, since TPC members would now be paid for their time. Although the financial incentive is not that great (e.g., my hourly rate for consulting is something like 5 times my regular salary), getting paid should encourage TPC members to take the process more seriously.

The only downside I can see is people who sign up for a slew of program committees and become "professional paper reviewers", but TPC chairs would clearly have to balance this against the research credentials of the people being asked to serve. Note that many journals impose author fees for publication of the paper, but presumably you are willing to pay once you have done all the work to get the paper accepted. And conferences expect authors to show up at the conference to present the paper, which can get to be pretty expensive as well. But it seems crazy to me that the research community provides this free paper reviewing service with no negative ramifications for submitting totally unpolished work.

The Paperless Tourist

2010-03-04T18:06:00.000-08:00

I recently spent a week in Portugal for EWSN'10 and spent a few days in Lisbon and Porto on either end of the conference. I decided this time to go entirely paperless -- that is, not take a paper guidebook. Rather, I was going to rely entirely on my iPhone for all of the travel information. As an experiment it was largely successful, with some caveats.

Normally I take a Rough Guide or Lonely Planet guidebook with me when I travel, but this has two big disadvantages. First, I have to lug the book around wherever I go, which usually means also having a bag or something else just to carry the book when I'm out on the town. Second, having the guidebook out in a bar, restaurant, or on the street immediately pegs you as a tourist and I hate being so conspicuous. I'm all about blending in, as the picture on the right should make absolutely clear. (Pop quiz: Which one is me? Hint: I don't smoke.)

This time, I decided to rely on the iPhone Kindle app and bought the Rough Guide to Portugal for Kindle. So the entire text of the book was in my pocket at all times, and reading the book on the iPhone just makes me look like another cell-phone-obsessed tech junkie, which is fine by me. (At least in most places that I travel, although I have been to some pretty dodgy places where messing with an iPhone in public is likely to garner some unwanted attention.)

The big disappointment was that the resolution of the maps in the Kindle Rough Guide is not good enough to actually read the street names and markers -- even when zooming in on the map. I am not sure if this is unique to the iPhone or whether I'd have the same problem on a proper Kindle (I don't have one so I can't tell). I can say it isn't a problem when using a paper guidebook. So I could not really use the maps in the guidebook at all.

Of course, Google Maps is great on the iPhone and the GPS feature is a huge help when you're trying to get your bearings. However, this requires use of your data plan, which is expensive overseas. I bought a 50MB international data add-on which costs about $60. Other than having to monitor my usage it was a pretty good investment, though I really wish it were not necessary.

There are a couple of iPhone apps allowing you to download maps for offline viewing, including OffMaps (and quite a few rip off apps that simply take the same data and package it for a single city and sell you that alone for 99 cents.) They also permit use of GPS without incurring data charges. Unfortunately, they use free map sources that are much less accurate and complete than Google Maps -- the map for Coimbra was just terrible and only showed a couple of major highways, and none of the smaller back streets of the city. So the quality varies a lot.

The best iPhone app by far was the Lonely Planet Lisbon City Guide, which includes a great map with all of the restuarants, bars, etc. listed and linked to a little page telling you about the place with its hours. You can even search the guide, unlike the Kindle app which has no search capability. It's pretty terse but for a few days in a city was more than adequate. I also like how Web links can be tapped directly -- in case you want to dip into your data plan, say to check out the website for a restaurant or hotel -- the same is true in the Kindle e-books as well.

The final caveat was the limited battery life of the iPhone. The Kindle app does not eat a lot of power (I've read for hours on it with hardly a dent in the battery) but use of the GPS is pretty energy-intensive. While traveling I was using the iPhone a lot more than I usually do, and found that by late afternoon or early evening I was getting into dangerous territory, necessitating a quick recharge at the hotel if I was going to last the evening. Fortunately this coincides with my usual tourist siesta so it was not a problem.

If it were not for the poor resolution of the maps in Kindle Rough Guide, this combination of apps would have been an ideal solution for travel without a paper guidebook. If they can just fix that I'm ready to conquer the world without a book.

EWSN 2010 Keynote Video

2010-02-22T13:55:00.000-08:00

I've posted the video for my keynote at EWSN 2010 below. You can check out the full resolution version at blip.tv.

Highlights from EWSN 2010

2010-02-22T09:39:00.000-08:00

I was invited to give the keynote speech at the European Wireless Sensor Networks conference in Coimbra, Portugal. This was a fantastic location for a conference -- Coimbra has one of the oldest universities in Europe, over 700 years old. It's a beautiful city. EWSN is the European counterpart to conferences such as SenSys and IPSN. It is a very different crowd than typically attends those events. I learned a lot about a couple of the big EU-sponsored sensor nets projects including CoNet and GINSENG. Interestingly, the Contiki OS seems to be pretty popular amongst the European research groups, in contrast to the TinyOS-dominated US landscape.

My keynote was entitled "The Next Decade of Sensor Networking" and I tried to argue that the field is running the risk of becoming stagnant unless we define some big research challenges that will carry us for the next decade. I've blogged about these themes here before. I delivered the talk in "Larry Lessig" style -- having written the "script" as an essay and then making slides to highlight the key points, rather than starting with the slides and ad libbing the voiceover as I usually do. I'll post a video here soon - the slides are more than 50 MB and don't really go well on their own.

A couple of highlights from the conference, though I had to miss the last day.

Jayant Gupchup from Johns Hopkins gave a talk on Phoenix, an approach to reconstructing the timestamps for sensor data after the fact. The idea is to not use a time synchronization protocol, but rather have nodes log enough data that can be used for post-hoc time correction. This is an interesting problem that was motivated by their experiences running sensor nets for more than a year, in which they observed a lot of node reboots (which throw off simple timing approaches) and extended periods when there was no suitable global timebase. The Phoenix approach collects information on nodes' local timestamps and beacons from GPS-enabled nodes at the base station, and performs a time rectification technique, similar to the one we developed for correcting our volcano sensor network data. Phoenix achieves around a 1 sec data accuracy (which is acceptable for environmental monitoring) even when the GPS clock source is offline for a significant period of time.

Raghu Ganti from UIUC gave a talk on "Privacy Preserving Reconstruction of Multidimensional Data Maps in Vehicular Participatory Sensing." The title is a bit unwieldy, but the idea is to reconstruct aggregate statistics from a large number of users reporting individual sensor data, such as their vehicle speed and location. The problem is that users don't want to report their true speed and location, but we still want the ability to generate aggregate statistics such as the mean speed on a given road. Their approach is to add noise to each user's data according to a model that would make it difficult for an attacker to recover the user's original data. They make use of the E-M algorithm to estimate the density distribution of the data in aggregate.

Although the paper considered a number of attacks against the scheme, I was left wondering about a simple binary revelation of whether a user had recently left their home (similar to PleaseRobMe.com). One solution is to delay the data reporting, although one would be able to learn the approximate time that an individual was likely to leave home each day. The other approach is to perturb the timing data as well, but this would seem to interfere with the ability to ask questions about, say, traffic levels at certain times of day.

Finally, Christos Koninis from the University of Patras gave a talk on federating sensor network testbeds over the Internet, allowing one to run testbed experiments across multiple testbeds simultaneously, with "virtual" radio links between nodes on different testbeds. So you could combine a run on our MoteLab testbed (around 190 nodes) with the TWIST testbed (220 nodes) to get a virtual testbed of more than 400 nodes. This is a very cool idea and potentially extremely useful for doing larger-scale sensor net experiments. Their approach involves routing data over a node's serial port through a gateway server to the other end where it is injected into the destination testbed at the appropriate point. They can emulate a given packet loss across each virtual link, not unlike Emulab. Unfortunately they did not really consider making the cross-testbed packet transmission timings realistic, so it would be difficult to use this approach to evaluate a MAC protocol or time sync protocol. It also does not properly emulate RF interference, but I think this is still a very interesting and useful ideas. Another cool aspect of this project is that they can add virtual simulated nodes to the test
bed, allowing one to run mixed-mode experiments.