I just got back from HotMobile 2014 (for which I was the general chair). HotMobile is the mobile systems community's "hot topics" workshop, held annually as a forum for (according to the Call for Papers) "position papers containing highly original ideas" and which "propose new directions of research" or "advocate non-traditional approaches". It's a small workshop (we had about 95 people this year) and the paper submissions are short -- 6 pages, rather than the regular 14.
Overall, the workshop was great -- lots of good discussions, good talks, interesting ideas. And yet, every time I attend one of these "hot topics" workshops, I end up feeling that the papers fall well short of this lofty goal. This is not limited to the mobile community -- the HotOS community has a similar problem as well.
This has bugged me for a long time, since it often feels as though there is no venue for doing "out of the box" work that is intended to look out five or ten years -- rather than just things that are incremental but not yet ready for publication in a major conference like SOSP or MobiSys. I also have fond memories of HotOS in the late 1990s in which it felt as though many of the papers were there to shake up the status quo and put forward a strong position.
What I've now come to realize is that there is a tremendous value in having a small workshop for preliminary (and often incremental) results. The community obviously feels that such a venue is useful, despite its lack of "hotness" -- we had a record number of attendees this year, and (I believe) a near-record number of submissions.
And after all, the main reasons to attend any workshop are the discussions and networking -- not the papers.
The problem is that we insist on calling this a "hot topics" workshop and pretend that it's about far-out ideas that could not be published elsewhere. Instead, I think we should be honest that HotMobile (and HotOS, HotNets, etc.) are really for three kinds of papers:
So, my humble proposal is to rename the workshop "ColdMobile" and, just to be cheeky, hold it at a ski resort in the winter.
 |
| The HotMobile'14 poster and demo session. Look how happy those mobile systems researchers are! |
This has bugged me for a long time, since it often feels as though there is no venue for doing "out of the box" work that is intended to look out five or ten years -- rather than just things that are incremental but not yet ready for publication in a major conference like SOSP or MobiSys. I also have fond memories of HotOS in the late 1990s in which it felt as though many of the papers were there to shake up the status quo and put forward a strong position.
What I've now come to realize is that there is a tremendous value in having a small workshop for preliminary (and often incremental) results. The community obviously feels that such a venue is useful, despite its lack of "hotness" -- we had a record number of attendees this year, and (I believe) a near-record number of submissions.
And after all, the main reasons to attend any workshop are the discussions and networking -- not the papers.
The problem is that we insist on calling this a "hot topics" workshop and pretend that it's about far-out ideas that could not be published elsewhere. Instead, I think we should be honest that HotMobile (and HotOS, HotNets, etc.) are really for three kinds of papers:
- Preliminary work on a new project which is not yet ready for a major conference. Getting early feedback on a new project is often very useful to researchers, so they know if they are barking up the right trees.
An example of this from this year is the CMU paper on QuiltView, which proposes allowing users to pose real-time queries ("How is the weather down at the beach in Santa Barbara?") and get back real-time video snippets (from users wearing Google Glass!) in reply. This work is no where near mature enough for a full conference, and I hope the authors gained something from the paper reviews and discussion at the workshop to shape their future direction. - An incremental, and possibly vestigial, step, towards the next major conference paper on a topic. Many such papers are simply not big enough ideas for a full conference paper, but make a nice "short paper" for the sake of getting some idea out there.
One example from this year is this paper on the dangers of public IPs for LTE devices. This isn't something that's going to turn into a longer, more pithy paper later on, but is probably worth reporting. - The odd wacky paper that falls under the "hot topics" rubric. These are increasingly rare. About the only example from this year is this Duke paper on adding smart capabilities to childrens' toys with smartphones -- but the idea is not that radical.
So, my humble proposal is to rename the workshop "ColdMobile" and, just to be cheeky, hold it at a ski resort in the winter.
Dibs on handling local arrangements for ColdMobile
ReplyDeleteI basically agree with you, but another way to look at it is to ask why aren't there enough high-quality truly out-of-the-box papers to fill a workshop? (I'm assuming that if such submissions existed, they would be accepted over type 1 and type 2.) Academia is supposed to be where the far-out stuff gets attention. Is that kind of work is not happening, then why are people doing systems research in academia?
ReplyDeleteI don't know how things look on that side of the Atlantic, but in EU big part of the problem is the fact that one can barely get such "hot" ideas publicly funded anymore. Each new iteration of the big funding schemes, like the EU framework programs (http://ec.europa.eu/programmes/horizon2020/), has progressively reduced the targeted "exploitation horizon". Blue-sky research funding is now limited only to a handful areas deemed important enough through special initiatives like the FET Flagships (http://cordis.europa.eu/fp7/ict/programme/fet/flagship/).
DeleteYou know why it was a record # of attendees right Matt?
ReplyDeleteHard to turn down a beachside visit in the midst of the polar vortex!
And I'm sure the great papers had something to do with it too ;-)
I think it's very difficult to have 30 papers in a specialized workshop every year that are truly radical, exciting, and potentially high impact in a future form. The fact is, most good research comes in small increments. And these increments might be truly exciting to a few people (I certainly get excited about each one of my projects) but probably seem more incremental to an outsider, even someone working in the same general area.
ReplyDeleteThat said, I would hope that the Hot* workshops are doing all they can to attract out-of-the-box work.
Also, I think it's worthwhile for PCs to draw a distinction between hot topics in the sense of "highly original ideas" and hot topics in the sense of "new papers on whatever is currently trendy". Both can be good contributions, but there is usually vastly more opportunity for the latter type to be published. Hence my proposal for CoolNets 2014.
ReplyDeleteI think more consideration should be given to the alternative hypothesis, which is that there were never any (or very many) "hot" papers in the sense this post is about. Lots of papers seem really cool to grad students that seem less cool to more senior people, and the past contained lots of boring incremental work.
ReplyDelete