How can academics do research on cloud computing?

This week I'm in Napa for HotOS 2011 -- the premier workshop on operating systems. HotOS is in its 24th year -- it started as the Workshop on Workstation Operating Systems in 1987. More on HotOS in a forthcoming blog post, but for now I wanted to comment on a very lively argument discussion that took place during the panel session yesterday.

The panel consisted of Mendel Rosenblum from Stanford (and VMWare, of course); Rebecca Isaacs from Microsoft Research; John Wilkes from Google; and Ion Stoica from Berkeley. The charge to the panel was to discuss the gap between academic research in cloud computing and the realities faced by industry. This came about in part because a bunch of cloud papers were submitted to HotOS from academic research groups. In some cases, the PC felt that the papers were trying to solve the wrong problems, or making incorrect assumptions about the state of cloud computing in the real world. We thought it would be interesting to hear from both academic and industry representatives about whether and how academic researchers can hope to do work on the cloud, given that there's no way for a university to build something at the scale and complexity of a real-world cloud platform. The concern is that academics will be relegated to working on little problems at the periphery, or come up with toy solutions.

The big challenge, as I see it, is how to enable academics to do interesting and relevant work on the cloud when it's nearly impossible to build up the infrastructure in a university setting. John Wilkes made the point that that he never wanted to see another paper submission showing a 10% performance improvement in Hadoop, and he's right -- this is not the right problem for academics to be working on. Not because 10% improvement is not useful, or that Hadoop is a bad platform, but because those kinds of problems are already being solved by industry. In my opinion, the best role for academia is to open up new areas and look well beyond where industry is working. But this is often at odds with the desire for academics to work on "industry relevant" problems, as well as to get funding from industry. Too often I think academics fall into the trap of working on things that might as well be done at a company.

Much of the debate at HotOS centered around the industry vs. academic divide and a fair bit of it was targeted at my previous blog posts on this topic. Timothy Roscoe argued that academia's role was to shed light on complex problems and gain understanding, not just to engineer solutions. I agree with this. Sometimes at Google, I feel that we are in such a rush to implement that we don't take the time to understand the problems deeply enough: build something that works and move onto the next problem. Of course, you have to move fast in industry. The pace is very different than academia, where a PhD student needs to spend multiple years focused on a single problem to get a dissertation written about it.

We're not there yet, but there are some efforts to open up cloud infrastructure to academic research. OpenCirrus is a testbed supported by HP, Intel, and Yahoo! with more than 10,000 cores that academics can use for systems research. Microsoft has opened up its Azure cloud platform for academic research. Only one person at HotOS raised their hand when asked if anyone was using this -- this is really unfortunate. (My theory is that academics have an allergic reaction to programming in C# and Visual Studio, which is too bad, since this is a really great platform if you can get over the toolchain.) Google is offering a billion core hours through its Exacycle program, and Amazon has a research grant program as well.

Providing infrastructure is only one part of the solution. Knowing what problems to work on is the other. Many people at HotOS bemoaned the fact that companies like Google are so secretive about what they're doing, and it's hard to learn what the "real" challenges are from the outside. My answer to this is to spend time at Google as a visiting scientist, and send your students to do internships. Even though it might not result in a publication, I can guarantee you will learn a tremendous amount about what the hard problems are in cloud computing and where the great opportunities are for academic work. (Hell, my mind was blown after my first couple of days at Google. It's like taking the red pill.)

A few things that jump to mind as ripe areas for academic research on the cloud:

• Understanding and predicting performance at scale, with uncertain workloads and frequent node failures.
• Managing workloads across multiple datacenters with widely varying capacity, occasional outages, and constrained inter-datacenter network links.
• Building failure recovery mechanisms that are robust to massive correlated outages. (This is what brought down Amazon's EC2 a few weeks ago.)
• Debugging large-scale cloud applications: tools to collect, visualize, and inspect the state of jobs running across many thousands of cores.
• Managing dependencies in a large codebase that relies upon a wide range of distributed services like Chubby and GFS.
• Handling both large-scale upgrades to computing capacity as well as large-scale outages seamlessly, without having to completely shut down your service and everything it depends on.

What I'm working on at Google: Making the mobile web fast

A bunch of people have asked me what I work on at Google these days. When I joined Google last July in the Cambridge office, I worked with the team that runs Google’s content delivery network, which is responsible for caching a vast amount of (mostly video) content at many sites around the world. It is a fantastic project with some great people. My own work focused on building tools to measure and evaluate wide-area network performance and detect performance problems. This was a great “starter project,” and I got to build and deploy some pretty large systems that now run on Google’s worldwide fleet.

Now that I’m in Seattle, I am heading my own team with the charter to make the mobile web fast. By “mobile web”, I mean the entire web as accessed from all mobile devices, not just Google services and not just Android. While Android is a big focus of our work, we care a lot about improving performance for all mobile devices. This project is an outgrowth of Google’s broader make the web faster initiative, which has (to date) largely been focused on desktop web. The parent project has been involved with the Chrome browser, the SPDY protocol (a replacement for HTTP), the WebP image format, numerous network stack enhancements, and more. I see mobile as the next big frontier for the web and there are some huge opportunities to make impact in this space.

This is a hugely exciting project since it touches on many different platforms and cuts across layers. Everyone knows that mobile web usage is taking off: the growth of the mobile web is much, much faster than the growth of the desktop web was back during the dot-com boom. At the same time, the mobile web is painfully slow for most sites and most users. And of course, not everyone has the benefit of using a bleeding-edge 4G phone with LTE network speeds of 25 Mbps, like I do (my current phone is an HTC Thunderbolt, and it’s awesome). For the vast majority of Internet users, a mobile phone will be the only device they ever interact with.

So what are we working on? At a high level we are planning to tackle problems in three broad areas: The mobile devices themselves; the services they connect to; and the networks that connect them. On the device side, we are looking at a wide range of OS, network stack, and browser enhancements to improve performance. On the service side, we are looking at better ways to architect websites for mobile clients, providing tools to help web developers maximize their performance, and automatic optimizations that can be performed by a site or in a proxy service. Finally, at the network layer we are looking at the (sometimes painful) interactions between different layers of the protocol stack and identifying ways to streamline them. There is a huge amount of work to do and I am really fortunate to work with some amazing people, like Steve Souders and Arvind Jain, on this effort.

Our goal is to share solutions with the broader web development community -- we hope to release most of our code as open source, as many other Google projects have done. I also plan to maintain strong ties with the academic research community, since I feel that there is a great opportunity to open up new avenues for mobile systems research, as well as leverage the great work that universities are doing in this space.

I think Google is in a unique position to solve this problem. And, of course, we are hiring! Drop me a line if you are interested in joining our team.

How to write a grant proposal for industry

I've recently had the pleasure of reviewing proposals for Google's Research Awards program. This is a huge program that gives away millions of dollars a year to a large number of university research projects ranging from machine vision to human-computer interaction to mobile systems. After spending eight years in academia struggling to get funding for my own research, it is quite nice to be on the other side of the table and be the one helping to give away the money, rather than begging for it.

First of all, this is nothing like reviewing proposals for, say, the NSF, where you have 15-20 page proposals and project sizes ranging from 3-5 years and anywhere from one to ten PIs. The Google proposals are (thankfully) short -- only 3 pages -- and generally ask for funding for a couple of grad students for a year, plus some funding for a summer month of a PI and maybe some equipment. So the individual grants are small, which in some sense is frustrating since it's hard to propose anything big and groundbreaking when it's just for a year. Essentially this means that most PIs are only asking for money for things they are already working on, rather than spearheading work in a new direction. (Arguably Google should be giving away more money to fewer schools for longer periods of time, but this is my personal opinion.) Also, NSF proposals are reviewed by a panel of (mostly) academics drawn from all over the place, who are supposed to be impartial. At Google, the reviewers are both researchers and software engineers who may or may not be working in areas related to the proposal itself.

Most of the proposals I saw left something to be desired. Far too many of them were asking for money for things that we already know how to do (like write an Android app for your pet project) or which have been done to death (like develop yet another mobile ad hoc routing protocol). On the other hand there were the rare proposals that had an exciting new idea and proposed to do something that Google was not about to go do on its own.

A few tips if you ever apply to this program (and I certainly encourage you to do so).

First, think about who the reviewers are. They are mostly not people like me. Most of the engineers at Google don't have a lot of experience reading and reviewing research proposals (let alone writing them), and many of them are not going to be in your immediate research area. Try to reach out and explain why your work is important at a broader level; the reviewers in this case are typically not your peers.

Second, think about why Google should fund this research. The key question I asked myself was not whether Google would benefit from the research, but rather why should Google fund a university to do this project, rather than just do it ourselves. If Google can hire a couple of engineers to solve a problem, I don't see any reason for us to fund a university to do it instead. On the other hand, if the university PIs are going to do something hard, or groundbreaking, or risky that Google would not have the time or resources to do, we should fund it.

There's also the related question of why should Google fund a research effort rather than another funding agency, such as the NSF. This one is a lot easier to answer: I know from experience that it's damned hard to get money from the NSF for many kinds of projects, and Google can help seed new research efforts that would be difficult to get off the ground otherwise. But if a project seems like it can and should be funded through another agency, that makes it less attractive from my perspective.

Third, try to get the exciting ideas up front. Most Googlers are extremely busy and probably won't spend as much time reading the proposals as you'd like them to. If you bury the lead it will be much harder for the reviewers to see the big idea and get excited about your work. It also helps to establish your credentials in the proposal itself -- not just your CV, but a paragraph or two in the main text saying who you are and why you are the right person to do this research is incredibly helpful (especially in the case when the reviewer is outside your area).

Finally, it always helps to have a champion at Google. If there is someone within the company that you know personally, who can vouch for your work and wants to work with you on a project, this helps tremendously. Having a grad student spend time at Google as an intern is a great way to make those connections.

It's just a guess, but I would not be surprised if other companies' research grants worked in much the same way. While I was at Harvard, I got a lot of funding through places like Microsoft Research, Intel, IBM, and Sun, all of which have fantastic university research programs. (Well, except for Sun, which no longer exists.) Of course, keep in mind that I don't speak for the rest of the Google Research Awards committee, and other reviewers very likely use different criteria than I do.

This is my personal blog. The views expressed here are mine alone and not those of my employer. 

The death of Intel Labs and what it means for industrial research

Intel recently announced that it is closing down its three "lablets" in Berkeley, Seattle, and Pittsburgh. I know a lot of people who work at the Intel Labs and in fact spent a year at the Berkeley lab before joining Harvard in 2003.  (I should be clear that not all of Intel Research is closing down -- just the lablets.) All of the researchers have been told to find new jobs, though some of them are getting picked up by Intel-sponsored research centers at the nearby Universities.

The Intel Labs were a fantastic experiment to rethink how industrial research should be done. They first started in 2001 under the model that full-time Intel researchers would work side-by-side with faculty and students from the nearby universities. All of the research was done under an open intellectual property model where results were co-owned by the university and Intel. In fact the labs were not inside of the Intel corporate network and operated largely autonomously from the rest of Intel. This allowed projects to be done seamlessly across the Intel/academic barrier and for students to come and go without restrictions on the IP.

Some fantastic work came out of the Labs. The Berkeley lab drove most of the early work on sensor networks and TinyOS, especially while David Culler and his various students were there. The Seattle lab developed PlaceLab (the precursor to WiFi based localization found in every cell phone platform today); WISP (the first computing platform powered by passive RFID); and lots of great work on security of wireless networks. The Pittsburgh lab did work on camera-based sensor networks, cloud computing, and robotics. All of these projects have benefitted tremendously from the close ties that the Labs had with the university.

Before the Labs opened, Intel Research was consistently ranked one of the lowest amongst all major technology companies in terms of research stature and output. I feel that the Labs really put Intel Research on the map by involving world-class academics and doing high-profile projects. They have attracted some of the top PhDs and offered a much more academic alternative to a place like, say, IBM Research.

I have no idea why Intel decided to close the labs. The official press release is devoid of any rationale, and obviously tries to spin the positive angle (the establishment of the university-based research centers which will replace the Labs). I've spoken with a number of the researchers there since the announcement, and have formed my own theories about why Intel is shutting them down. The most obvious possibility is that the Labs are incredibly expensive to run, and it's hard to link the work they do to Intel's bottom line. After all, very little of the work done at the Labs is picked up by Intel's product groups. The Labs' mission has always been to inform the five-to-ten-year roadmap for the company. It's unclear to me whether they have been successful in this, though at least they have inspired some entertaining commercials.

Personally, I'm worried about what this means for industrial computer science research. Here is one of the world's largest and most wealthy tech companies, closing down a set of labs that employs some of the top minds in the field, which by all measures has been really successful in producing novel and high-impact research. If Intel can't figure out how to leverage that amazing talent pool, it does not bode well for the rest of the industry.

Maybe this suggests is that the conventional industrial research model is simply broken. The only (important) places left that use this model are Microsoft, IBM, and HP.  These companies can afford to set up big labs with lots of PhDs and pay them to do whatever the hell they want with little accountability, but maybe this model is no longer sustainable. As I've written before, Google takes a very different approach, one in which there is no division between "research" and "engineering." The advantage is that it's always clear how the research activities relate to the company's priorities, although it does mean that researchers are not doing purely "academic" work, the main output of which is more papers.

One closing thought. Perhaps Intel realizes it can have far more impact by setting up large, high-impact research programs within universities rather than run its own labs. In some ways I can appreciate this point of view: help the universities do what they do best. But the way this is being done is unlikely to be successful. The first such Intel center on visual computing involves something like 25 PIs spread across eight universities. Each PI is only getting enough to fund work they were already doing, so this is an example of doing something that looks good on paper but is unlikely to move the needle at all for these research groups. This seems like a missed opportunity for Intel.

Obligatory disclaimer: This is my personal blog. The views expressed here are mine alone and not those of my employer.

Carriers are not ready for tablets

This week I spent at least two hours on the phone trying to convince both AT&T and Verizon to give me online access to accounts I set up for tablets that I am testing -- the Samsung Galaxy Tab (AT&T) and Motorola Xoom (Verizon). They are both great devices; I like the Galaxy Tab's form factor (like a paperback book) and the Xoom is incredibly fast. But it is clear that the wireless carriers have no idea how to incorporate these devices into their billing and customer service ecosystem. It was such a painful and frustrating experience that I wonder how the cellular carriers expect to leverage these devices as more tablets come onto the market.

First, my story with AT&T. I bought the Galaxy Tab a few months ago which came with an AT&T SIM card pre-installed. When you boot the device for the first time, there's a widget which takes you to a registration page, which I filled out to activate the tablet on AT&T's network. Since then I have not received a bill (to my knowledge) for the usage, and couldn't remember whatever password I might have used to set up the device. Nothing on AT&T's website seemed to offer any help, as it is completely oriented towards phones.

Finally, I gave up and called AT&T to re-register the Tab manually. The first customer service rep had no idea how to do this. They kept asking for the phone number, which the device does not have (at least when you enter the Settings menu it lists the phone number as "unknown"). Given that I had not received any bills I suspected the Tab was never registered, so we had to look it up by IMEI number. The rep could not pull up any account information. She ended up transferring me to technical support at Samsung, of all places. The Samsung rep was very friendly but couldn't help with this problem, either -- it seemed to be an AT&T issue (and I agree). I ended up having to call AT&T back, and went through the same painful process of explaining what my problem was. This rep ended up transferring me over to a different sales rep who tried to help me set up the account from scratch.

This is when things started to go downhill. All I wanted as an unlimited (or as close as possible) data plan for the Galaxy Tab. I could see online that AT&T has a 2GB/month data plan for tablets but the rep kept telling me that "their internal systems don't necessarily show what is on the website." (First warning sign there.) Eventually he managed to pull up the right plan but couldn't seem to figure out how to add a Galaxy Tab -- the device wasn't showing up in his menus. It sounded like he had never activated a tablet before. After around 20 minutes on hold he managed to figure it out, so I think I finally have the Galaxy Tab set up for data access. I was promised that I would get an email from AT&T confirming the new account, but it never arrived. So I guess I am going to have to call them back. I am dreading this.

Verizon was almost as bad. Like the Tab, I had set up the Xoom using the registration app on the tablet itself. I had made a note of the username used to set up the account, but not the password. Verizon's website offers no ability to request a new password except via SMS to the device -- and the Xoom doesn't receive SMS messages, since it's not a phone. The only way to request a new password is to spend around 45 minutes on the phone with various Verizon reps with the result being that a new password is being sent to me by postal mail in five business days. (What is this, the nineteenth century?) Of course, since I moved recently, my mailing address on file with Verizon was incorrect. Fortunately, the service rep allowed me to change the postal address over the phone -- meaning that they trust me enough to let me change my mailing address, not not enough to reset my online account password. This makes absolutely no sense and seems designed to drive users away.

The lesson here is that the wireless carriers have no clue how to incorporate tablets. They are treating them like phones, which they aren't.

Thinking back on 8 years in Boston

Tomorrow I will be packing up and moving from Boston to Seattle with my family. I thought now would be a good time to reflect on living in Boston as a city and recall some of my best memories here.

I've never lived in Seattle, though have been there many times -- it seems like a wonderful city, full of funky crazy people and absolutely beautiful geography. I'm not terribly excited about the rainy weather, though something tells me it can't be any worse than the Boston winters, when I always feel cooped up. I really miss getting out to go hiking with the dog or mountain biking during the winter months in New England -- and now that I have a kid it's especially hard to get out when it's well below freezing outside (he has a lot lower tolerance for the winter weather than I do). Rain I can deal with; negative 20 wind chills and a foot and a half of snow are something different altogether.

On finishing grad school at Berkeley in 2002, I had a few faculty job offers, and my wife was looking for residency programs in psychiatry. Our decision came down to two cities: Boston, where I had an offer at Harvard, and Pittsburgh, where I had an offer at CMU. I had lived in Boston for a while during college, so I knew I liked the city. But CMU was a very tempting offer, being a much more highly-ranked CS program than Harvard. My wife and I visited Pittsburgh a couple of times and actually liked it a lot: it was a very friendly place, and the CMU folks went all out to show us a good time. At one point we actually made the decision to move to Pittsburgh but decided to sleep on it. The next day we had to ourselves, without anyone showing us around. The only Mexican place that served "fish tacos" appeared to be Van De Kamp's fish sticks on a tortilla. I could not find a music store that allowed you to browse the CDs without an attendant unlocking a glass case to let you inside. We tried to find a decent shopping mall, hoping they would have a good record store, but found ourselves in the mall where the movie Dawn of the Dead was filmed -- I did not make it more than three paces beyond the door before we realized it was a terrible mistake. I'm sorry, Pittsburgh might be a wonderful city for some people, but it was not for us.

Monroeville Mall on a typical Sunday afternoon.

So we moved to Boston in 2003. We drove across the country in my little beat-up two-door Ford, stopped at places like Moab and St. Louis and Nashville, a little like On the Road in reverse. We arrived on a hot, sticky summer day in a thunderstorm and moved into an apartment on Dana Street in Cambridge, not far from Harvard Square. Almost immediately we felt like outsiders. Boston is a very old city, and it shows -- the old brick buildings around Harvard, the beat-up sidewalks, everything dripping with history and significance. Paul Revere. George Washington. Boston Common. Old Granary Burial Ground. The feeling was so different than the newness that is so pervasive in California. It took some getting used to.

These gravestones have been here for a while. (From http://www.flickr.com/photos/harvardavenue/66097831/)

That first summer was one of the most exciting in Red Sox history. I had never paid attention to baseball before, but it soon became clear that if I wasn't conversant in Curt Schilling or Manny Ramirez I was going to be left out of a lot of conversations. Like a lot of people in Boston, I got caught up in the excitement of the 2003 postseason when the Sox were narrowly beat by the Yankees for the ALCS title. The next year was even more exciting, in that the Sox went on to win the World Series for the first time in 86 years. The whole city went totally apeshit. For so many people living here, it was like a moment of rapture that they never believed would actually happen, the culmination of a lifetime of inferiority to cities like New York. I feel sad for some long-time Bostonians who now have no excuse to be cynical about their station in life.

People were just a little excited. (From http://www.flickr.com/photos/eandjsfilmcrew/412103968/)

Not long after we moved to Boston they finally banned smoking in bars and restaurants (which we'd been used to in California) and actually allowed alcohol sales on Sundays. It was as if the city were modernizing before our eyes. It would take another six years for Cambridge to allow outdoor dining at restaurants, which is still encumbered with backwards vestiges of the Puritans (you can only have a drink while sitting outside if you also order food).

Going out in Boston was a bit more a dressy, formal affair than we were used to in Berkeley. At all but the very highest end restaurants in San Francisco, jeans and a t-shirt were acceptable attire; not so in Boston. On the positive side, Boston has a great foodie scene. At first we were totally lost trying to find good places to eat; Zagat's is useless and Yelp simply reflects the lowest common denominator. At first we were convinced that people in Boston had no idea what real, good Mexican food was -- those greasy platters of "enchiladas" covered in melted cheese that are so popular at hellholes like Casa Romero are not it. Then we discovered Chowhound, and got turned on to a whole world of hole-in-the-wall places serving authentic Mexican and Salvadorean and Sichuan and Cambodian. Unlike Berkeley, where you can swing a dead cat and hit three burrito joints and a place with out-of-this-world sushi, in Boston it takes a bit more digging, but there is great food here.

Some of my best memories of living in Boston...

Walking my dog, Juneau, to work at Harvard every day, stopping at the coffee shop on the way, and taking her to the dog park on the way home for an hour or so of play time with the other dogs.

Juneau would occasionally help me reviewing papers, too.

Sitting outside on a warm summer evening, firing up the grill, having friends over for dinner and drinks until late.

Every single fall, feeling the first day of cold air and getting excited for the leaves to start changing.

This image has not been enhanced.

Riding my bike along the banks of the Charles River, whizzing by rollerbladers and clueless tourists walking four abreast in the middle of a bike path.

The morning after a big snowstorm, seeing the world transformed and noticing how quiet everything was with the snow on the ground.

Shoveling is always fun too.

Late nights out in Boston Chinatown with Gu, drinking beer and eating Korean food.

Watching the sun rise out of the window of Mount Auburn hospital on a hot July day a couple of hours before my son was born.

I was a dad not long after this picture was taken.

Running a successful program committee

Yesterday we held the program committee meeting for the 13th Workshop on Hot Topics in Operating Systems (HotOS), for which I am serving as the program chair. This is the premier workshop in the OS community and focuses on short (five page) position papers meant to bring out exciting new research directions for the field. In some years it has been more exciting than others. What tends to happen is that people send five-page versions of an SOSP submission they are working on, which (in my opinion) is not the best use of this venue. When HotOS becomes an SOSP preview I think it misses an important opportunity to discuss new and crazy ideas that would not make it into a regular conference.

We accepted 33 out of 133 submissions. The number of accepted papers is a bit higher in previous years because I wanted to be more inclusive, but also recognized that we could fit more presentations in at the workshop when you don't have 25-minute talk slots. There is no reason a 5-page paper needs a 25-minute talk, and I think it goes against the idea of the workshop to turn it into a conventional conference.

This was, by far, the best program committee experience I ever had, and I was reflecting on some of the things that made it so successful.

I've been on a lot of program committees, and sometimes they can be a very painful experience. Imagine a dozen (or more) people crammed into a room, piles of paper and empty coffee cups, staring at laptops, arguing about papers for 9 or 10 hours. Whether a PC meeting goes well seems to be the result of many factors...

Pick the best people. We had a stellar program committee and I knew going in that everyone was going to take the job very seriously. Everyone did a fantastic job and wrote wonderful and thoughtful reviews. These folks were invested in HotOS as a venue, were the kind of people who often submit papers to the workshop, and care deeply about the systems community as a whole. The discussion at the PC meeting itself was great, nobody seemed to get cranky, and even after 8+ hours of discussing papers there was still a lot of energy in the room. This is helped a lot by the content -- HotOS papers tend to be more "fun" and since they are so short, you can't nitpick every little detail about them.

Set expectations. I tried to be very organized and made sure that the PC knew what was expected of them, in terms of getting reviews done on time, coming to the PC meeting in person, what my philosophy was for choosing papers, and how we were going to run the discussion at the meeting. I think laying out the "rules" up front helps a lot since it keeps things running smoothly. I've blogged about this before but I think establishing some ground rules for the meeting is really useful.

Get everyone in the room. Having a face-to-face PC meeting is absolutely key to success. Everyone came to the PC meeting in person, except for one person whose family fell ill at the last minute and had to cancel, but even he phoned in for the entire meeting (I can't imagine being on the phone for more than eight hours!). I made sure the PC knew they were expected to come in person, and nailed down the meeting date very early, so everyone was able to commit. Letting some people phone in is a slippery slope. I can't count how many PC meetings I've been to that have been hampered by painful broken conference call or Skype sessions.

Use technology. We used HotCRP for managing submissions and reviews, which is by far the best conference management system out there. During the PC meeting itself, I shared a Google spreadsheet with the TPC which had the paper titles, topic area, accept/reject decision, and a one-line summary of the discussion. The summary was really helpful for remembering what we thought about a paper when revisiting it later in the day. Below is a snippet (with the paper numbers and titles blurred out). The "order" column below is the order in which the paper was discussed. This way, everyone in the PC could see the edits being made in real time and there was rarely confusion about which paper we were discussing next.

Pre-reject and pre-accept. I rejected around half of the submissions before the PC meeting and gave the PC a chance to revive any such paper for discussion (none were). I also "pre-accepted" about 10 papers that were noncontroversial; we saved discussion of these for the end of the day, since they were easy cases. We ended up discussing a total of 69 papers at the meeting, which meant we had to go at a pretty good clip.

Be definitive.  With very few exceptions, we tried to reach a clear accept/reject decision on each paper as we discussed it, and did not table any papers for later discussion. There was one case where we were hung on what to do with a paper and decided to push the discussion until the end of the day. In cases where there was disagreement, I would mark a paper as "presumed reject" or "presumed accept" and put down the name of the person who wanted to argue for the opposite outcome later. That gave us a chance to move on when there was an insurmountable debate, and it was clear that the champion (or anti-champion) of a paper would have a chance to have their say.

Take everyone out to a nice dinner afterwards. As far as I'm concerned, this was the best part of hosting the PC meeting.