Skip to main content

SOSP 2009, Day Three

Final day here at SOSP 2009. Unfortunately my shuttle back to the airport leaves in the middle of the first session, so I am only able to go to the first couple of talks.

Last night at the SIGOPS business meeting, Doug Terry gave an overview of SIGOPS' finances; apparently they are making money hand over fist, and are actually looking for ways to spend down some of the savings. My suggestion: host SOSP 2011 in the ideal location for the conference and don't worry about breaking even.

Speaking of SOSP 2011, Peter Druschel was announced as the Program Chair. There were a few proposals for sites to host the conference. The one that received the most applause was Madeira Island, which is some 500 miles off the coast of Portugal. The site looks fantastic, though I'm not sure about travel connections. Lisbon was the other Portuguese option, which looks great. There were proposals for Dresden, China, Brazil, New Hampshire, and Puerto Rico as well.

The rest of the business meeting included discussions of ACM's copyright policy (and whether SIGOPS should put pressure on them to change it to permit more open distribution of papers); a rehashing of the old single-blind-vs-double-blind controversy (thankfully short); and some discussion of whether the SIGOPS Hall of Fame award should be opened up to non-peer-reviewed publications (there was widespread support for this).

The only talk I have time to write up today is...

Distributed Aggregation for Data-Parallel Computing: Interfaces and Implementations

This paper deals with data-parallel computation in clusters, popularized by systems such as MapReduce and Hadoop. Microsoft's DryadLINQ system automatically generates distributed queries on clusters from application code (e.g., in C#). In this paper, the authors explore programming interfaces to enable distributed aggregation within the cluster: rather than the conventional two-phase "map-reduce" paradigm, allowing intermediate aggregation at source nodes and within intermediate nodes. Doing so can greatly reduce disk and network I/O. The talk emphasized that the programming interface for defining iterator functions can have a substantial impact on performance, and the authors have experimented with a range of aggregation and pipelining strategies.

I have to run to the airport, so I don't have time to write up the Quincy presentation in detail, but kudos to the authors for the Zero Punctuation-themed talk. I'm bummed that I have to miss the UpRight talk as well as the entire last session.


  1. Michael gets all the credit for the Quincy presentation. You can render it in your browser if you have Silverlight (latest version?) installed: Click anywhere to advance the animations. Might be best to watch the video of the talk at the same time:


Post a Comment

Popular posts from this blog

Why I'm leaving Harvard

The word is out that I have decided to resign my tenured faculty job at Harvard to remain at Google. Obviously this will be a big change in my career, and one that I have spent a tremendous amount of time mulling over the last few months.

Rather than let rumors spread about the reasons for my move, I think I should be pretty direct in explaining my thinking here.

I should say first of all that I'm not leaving because of any problems with Harvard. On the contrary, I love Harvard, and will miss it a lot. The computer science faculty are absolutely top-notch, and the students are the best a professor could ever hope to work with. It is a fantastic environment, very supportive, and full of great people. They were crazy enough to give me tenure, and I feel no small pang of guilt for leaving now. I joined Harvard because it offered the opportunity to make a big impact on a great department at an important school, and I have no regrets about my decision to go there eight years ago. But m…

Rewriting a large production system in Go

My team at Google is wrapping up an effort to rewrite a large production system (almost) entirely in Go. I say "almost" because one component of the system -- a library for transcoding between image formats -- works perfectly well in C++, so we decided to leave it as-is. But the rest of the system is 100% Go, not just wrappers to existing modules in C++ or another language. It's been a fun experience and I thought I'd share some lessons learned.

Why rewrite?

The first question we must answer is why we considered a rewrite in the first place. When we started this project, we adopted an existing C++ based system, which had been developed over the course of a couple of years by two of our sister teams at Google. It's a good system and does its job remarkably well. However, it has been used in several different projects with vastly different goals, leading to a nontrivial accretion of cruft. Over time, it became apparent that for us to continue to innovate rapidly wo…

Running a software team at Google

I'm often asked what my job is like at Google since I left academia. I guess going from tenured professor to software engineer sounds like a big step down. Job titles aside, I'm much happier and more productive in my new role than I was in the 8 years at Harvard, though there are actually a lot of similarities between being a professor and running a software team.

I lead a team at Google's Seattle office which is responsible for a range of projects in the mobile web performance area (for more background on my team's work see my earlier blog post on the topic). One of our projects is the recently-announced data compression proxy support in Chrome Mobile. We also work on the PageSpeed suite of technologies, specifically focusing on mobile web optimization, as well as a bunch of other cool stuff that I can't talk about just yet.

My official job title is just "software engineer," which is the most common (and coveted) role at Google. (I say "coveted&quo…