Sunday, April 21, 2013

The other side of "academic freedom"

My various blog posts about moving from academia to industry have prompted a number of conversations with PhD students who are considering academic careers. The most oft-cited reason for wanting a faculty job is "academic freedom," which is typically described as "being able to work on anything you want." This is a nice theory, but I think it's important to understand the realities, especially for pre-tenure, junior faculty.

I don't believe that most professors (even tenured ones) can genuinely work on "anything they want." In practice, as a professor you are constrained by at least four things:
  • What you can get funding to do;
  • What you can publish (good) papers about;
  • What you can get students to help you with;
  • What you can do better than anyone else in the field.
These are important limitations to consider, and I want to take them one by one.

Funding doesn't come easy. When I was a PhD student at Berkeley, I was fortunate to be a student of David Culler's, who had what seemed like an endless supply of funding from big DARPA and NSF grants, among others. When I went to start my faculty career, he (and many others) told me I would have "no problem" getting plenty of funding. This turned out not to be true. Shortly after I started my faculty job, DARPA all but shut down their programs in computer science, and NSF grants became heavily constrained (and much more competitive). Being a freshly-minted faculty member meant I was essentially a nobody, but that didn't mean that NSF review panels took pity on me -- apart from special programs like the CAREER award, you're competing with the most senior, established people in your field for every grant. To make matters worse, I didn't have a lot of senior colleagues in my area at Harvard to write proposals with, so I mostly had to go it alone.

Now, I will readily admit that I suck at writing grants, although according to my colleagues my hit rate for funding was about on par with other profs in my area. However, there were several projects that I simply could not do because I couldn't get funding for them. I tried for four years to get an NSF grant for our work on monitoring volcanoes with sensor networks -- which was arguably the thing I was most famous for as a professor. I failed. As a result we never did the large-scale, 100-node, multi-month study that we had hoped to do. It was a huge disappointment and taught me a valuable lesson that you can't work on something that you can't get funding for.

Who decides which problems are sexy (and therefore publishable)? I'll tell you: it's the 30-some-odd people who serve on the program committees of the top conferences in your area year after year. It is very rare for a faculty member to buck the trend of which topics are "hot" in their area, since they would run a significant risk of not being able to publish in the top venues. This can be absolutely disastrous for junior faculty who need a strong publication record to get tenure. I know of several faculty who were denied tenure specifically because they chose to work on problems outside of the mainstream, and were not able to publish enough top papers as a result. So, sure, they could work on "anything they wanted," but that ended up getting them fired.

Now, there are some folks (David Culler being one of them) who are able to essentially start new fields and get the community to go along with them. I argue that most professors are not able to do this, even tenured ones. Most people have to go where the funding and the publication venues are.

What can you get students to work on? I don't mean this in a kind of grad-students-won't-write-unit-tests kind of way (although that is also true). What I mean is how likely is it that you will find grad students in your field who have the requisite skills to undertake a particular research agenda? In my case, I would have killed for some students who really knew how to design circuit boards. Or students who had some deep understanding of compiler optimization -- but still wanted to work on (and publish) in the area of operating systems. A bunch of times I felt that the problems I could tackle were circumscribed by my students' (and my own) technical skills. This has nothing to do with the "quality" of the students; it's just the fact that PhD students (by definition) have to be hyper-specialized. This means that grad students in a given area tend to have a fairly narrow set of skills, which can be a limitation at times.

Can you differentiate your research? The final (and arguably most important) aspect of being successful as a faculty member is being able to solve new problems better than anyone else in your area. It is not usually enough to simply do a better job solving the same problem as someone else -- you need to have a new idea, a new spin, a new approach -- or work on a different problem. Hot areas tend to get overcrowded, making it difficult for individual faculty to differentiate themselves. For a while it felt like everyone was working on peer-to-peer networking. A bunch of "me too" research projects started up, most of which were forgettable. Being one of those "me too" researchers in a crowded area would be a very bad idea for a pre-tenure faculty member.

Do things get better after tenure? I didn't stick around long enough to find out, so I don't know. I definitely know some tenured faculty who are coasting and care a lot less about where and how much they publish, or who tend to dabble rather than take a more focused research agenda post-tenure. Certainly you cannot get fired if you are not publishing or bringing in the research dollars anymore, but to me this sounds like an unsatisfying career. Others -- like David Culler -- are able to embark on ambitious, paradigm-shifting projects (like NOW and TinyOS) without much regard to which way the winds are blowing. I think most tenured faculty would agree that they are subject to the same sets of pressures to work on fundable, publishable research as pre-tenure faculty, if they care about having impact.

Okay, but how much freedom do you have in industry? This is worth a separate post on its own, which I will write sometime soon. The short version is that it depends a lot on the kind of job you have and what kind of company you work for. My team at Google has a pretty broad mandate which gives us a fair bit of freedom. But unlike academia, we aren't limited by funding (apart from headcount, which is substantial); technical skills (we can hire people with the skills we need); or the somewhat unpredictable whims of a research community or NSF panel. So, yes, there are limitations, but I think they are no more severe, and a lot more rational, than what you often experience as an academic.



47 comments:

  1. With all due respect, I feel that the case of a newly minted PhD would be a lot different than yours (a tenured professor moving into a relatively senior position at Google).

    How much freedom would the new PhD get if they were starting at Google? Also if the person was in a more abstract field, would they still have the same amount of freedom as a person in the more directly applicable field of operating/distributed systems?

    ReplyDelete
    Replies
    1. As I said, this is really deserving of its own (much longer) post. The amount of freedom you have at Google is the product of many factors. Seniority only plays a small role. One of the newly-minted PhDs on my team happens to be one of the tech leads and is defining the whole architecture of our software. This is because (a) he has the right level of technical background and (b) he's a real go-getter.

      Delete
    2. There's another important aspect of academic freedom that you can pretty much only get in academia: the freedom to speak your mind, in public, as an "expert" rather than having everything you say preprocessed by the reader's perception of your employer.

      Most academics don't use this freedom very much, but it's been a defining element of my own career.

      Delete
    3. Dan - that's very true. Tends to be more relevant for people working in the security and privacy area, as you do.

      Delete
    4. As a former Googler, I seriously disagree with your comment about freedom at Google to pursue one's interest at the non-senior level (as a general statement).

      It is generally *impossible* for a new phd to enter Google in a specific team and after a short while decide that they want to work on a new unrelated project, for which there is no headcount. The freedom that you mentioned of "defining a new architecture" is not really a freedom - in the same as the freedom to move to a different project. Not so in academia (though risks do exist).

      I agree that things may be different at a senior level.

      Delete
    5. Did I say anything about people at Google having complete freedom at any level?

      Delete
    6. I thought that was implied (considerable freedom at Google to select projects at a junior level). I'm glad you clarified it.

      Delete
  2. >>I tried for four years to get an NSF grant for our work on monitoring volcanoes with sensor networks

    Is it possible to do crowdsource funding of research project to cover the cost of hardware?

    Or is NSF/DARPA or other prestigious source of grants needed to get tenured. In other words, does the money matter where it came from?

    ReplyDelete
    Replies
    1. I am not sure how crowdsourcing would work in a university setting. There are rules about what kind of funding you can accept (e.g., for tax purposes), although private companies do fund university research under various terms, so I suppose it is possible. I don't know what the funders of a crowdsourced university effort would get out of it though. It's not like most university research projects yield a usable product. And then there are issues with IP rights and such. So, interesting idea, but the devil is in the details.

      More to the point: tenure committees do like to see that you can compete successfully for funding from NSF and other federal agencies, since that says something about the peer review of your work. So even if you have some external source of private funding, it's good to have at least some NSF money (for example).

      Delete
    2. There's a Y-Combinator backed startup called Microryza working on crowdfunding for research. The largest Microryza project I know of raised $15K (see http://blog.microryza.com/a-case-study-the-wormfree-world-campaign/ ) so it won't pay for a fully loaded grad student stipend any time soon. That said it can pay for some expenses.

      My girlfriend recently used it to fund research into why jokes are funny:
      https://www.microryza.com/projects/crowdsourcing-humor-using-humans-and-computers-together-to-write-jokes

      Delete
    3. Yep, like David mentioned, it seems *really* hard to raise six-figure grants of the sort that would fund one grad student for multiple years (or multiple grad students) via crowdfunding venues.

      Also, Matt might have alluded to this point, but at the department head and dean level, "official" grants from NSF, DARPA, etc. are more valued since the university collects overhead (a "cut" of your grant), whereas if you raised money from industry gifts or crowdfunding, the university doesn't see any of that money. I've heard that those sorts of funding "count" less on your CV.

      Delete
    4. I have to add here, that while NSF funding rates are ridiculous and still dropping, DARPA has turned the corner and recovered somewhat from the "Tether" years. They are now sponsoring entire programs managed by more "theoretical" and foundational program managers. The competition is still of course there, but much better than NSF, and with much larger payoffs (multi-million USD over 4-5 years), which free up faculty from the type of restrictive grant writing that Matt is referring to.

      My personal experience has not been nearly as bad as Matt's comments. I think NSF funding rates are tough, but you can squeeze out enough $ to fund a core of 2-3 students consistently over multiple years. Add to that gift funding opportunities from Google :), Cisco, HP etc, and the occasional non-NSF government entity, and life is much better than it was 6-7 years ago (when DARPA was driven by 6 month go-no-go tests, industry was in recession-defense mode, and NSF was the only game in town).

      Delete
    5. I'm glad to hear that DARPA is starting to fund CS research again.

      I was able to run a group with 5-6 PhD students, 3-4 undergrads, and (at one point) a couple of postdocs all at the same time. So while (for the most part) adequate funding was available, I had to spend an INSANE amount of time working to get it. And I still claim that having to compete for funding does restrict what you work on; you can't get funding to do just anything.

      Delete
    6. One indeed cannot get funding to do anything.

      However, this is not for lack of funding. Consider this, CISE funding has been going up steadily and significantly since we've been graduate students. That funding rates have gone down in some directorates is more about the size of academia increasing than funding going down. Moreover, as Ben points out DARPA is back, and there is significant CS funding now from DHS, IARPA, DOE, ONR and AFOSR among others. Some fields have it harder than others, but in general CS is in _much_ better shape than most of academia (including most of engineering and most of science, medicine aside).

      I think one of the biggest problems for young faculty is not getting good metoring about how the funding process works and in how grants need to be written to reflect that reality. Writing an NSF grant bears little resemblance to writing a DARPA grant which is nothing like writing a paper. Moreover, just as a paper for SIGCOMM ends up being written differently than a paper for SOSP, different funding programs (and PMs for that matter) at a given agency demand a different focus. This is the kind of stuff that people like David, Randy and Dave at Berkeley all get instinctively, but is completely opaque as a young faculty member. When looking for junior academic jobs I think people should always be asking questions "who will be my mentor?" and "what is my academic support system here?" There is no book for this stuff...

      .

      Delete
    7. I would like to point out that if "Dave at Berkeley" gets "this" instinctively, the "this" would be knowing that a *different* focus is demanded, not instinctively knowing exactly *what* focus is needed in all circumstances. A few years back he mentioned at a faculty retreat that he would have NO IDEA how to write a successful NSF proposal nowadays, because he'd been doing all his fundraising with industry lately.

      The giants are giants not only for what they know, but self-knowledge about what they don't!

      Delete
  3. I'll be eagerly waiting for your "longer blog post" about what freedom is like in industry. I'm one of those new PhD students looking down the path and wondering where I'll be in 6-8 years!

    ReplyDelete
    Replies
    1. My unsolicited advice is: Don't prematurely optimize :)

      The tech world will look *very* different by the time you finish your Ph.D. (heck, I started my Ph.D. before the iPhone and Facebook Apps came out, and I'm not even that old), so it's impossible to predict whether industry or academia will be the best option for you in 6-8 years.

      Throughout your Ph.D., just focus on doing great work that you're proud of, and hack on some fun side projects too!

      Delete
    2. Ah! First of all, I loved "The PhD Grind", and the Online Python Tutor is a thing of beauty.

      Second, that's true, and it's a calming thing to hear, especially since I'm hearing some good things about Education in america.

      However, I do have to be worried. The kinds of research I do now do have some impact on the types of jobs that will be open to me. My research interest is Digital Education (both CS Ed and otherwise), but that's looked down upon by most other research areas. I mean, I can blend my work with other areas and avoid branding myself as an Education researcher, but I'm not really sure that's what I want to be doing for the next 14 years of my life until I can get tenure...

      Delete
    3. I would tend to agree that "digital education" sounds like a very risky area to work on as a pre-tenure faculty member, unless you can find just the right kind of institution to support you.

      Delete
  4. Two separate points, from a post lightly edited from the Hacker News version:

    1. I'm surprised that you argue that tenured CS profs still have to follow what publication venues want to see. Why not just say, "Fuck it?", publish on blogs / arvix.com/, and let the field catch up to them? Certainly that's not a route to immediate promotions or status within the field, but there may be strong long-term returns to individuals who go this way and are vindicated over time.

    (This obviously doesn't apply to non-tenured faculty or grad students. I'll also note that this point is a related observation, not a criticism of his argument.)

    2. This stands out to me:

    The final (and arguably most important) aspect of being successful as a faculty member is being able to solve new problems better than anyone else in your area. It is not usually enough to simply do a better job solving the same problem as someone else -- you need to have a new idea, a new spin, a new approach -- or work on a different problem.

    Genuinely new ideas are actually quite rare. Sometimes the difference between a "new" idea and someone else's discovery or implementation of that idea can be just a couple months difference! (See Steven Berlin Johnson's Where Good Ideas Come From for one popular description of this.) Yet one person or group gets 99% of the credit / tenure / etc.

    ReplyDelete
    Replies
    1. Good points.

      As for #1 - even tenured faculty (usually) care about having impact, which means publishing in top venues. Even if they don't care about publications for their own sake, tenured faculty should care about helping their grad students get a good job upon graduation -- which usually means getting lots of conference papers. So it's quite difficult to say "fuck it" unless you don't care about impact and don't care about the careers of your students. That said, I have seen a number of tenured faculty who do exactly this.

      My point about a "new spin" is that often academics do wacky things to try to make their work stand out from the crowd, even if it isn't necessary to solve the problem at hand. Witness the whole programming languages community, which (in my opinion) is far too enamored with ever-more-esoteric abstractions rather than trying to solve real problems for real software developers.

      Delete
    2. For point #1, even if you cannot be fired after getting tenured, the department may have other means to make you feel "uncomfortable" (even "very uncomfortable") if your productivity (mostly measured in terms of papers and fundings) drops significantly. For one, for many universities, the official teaching load could be very high (4+4 for example). If you do not produce, the department may increase your teaching load to a level that pretty much makes you a teaching faculty.

      Delete
  5. A couple other limitations not directly mentioned here, that I've run into in my academic history:

    (1) You can't work on classified problems (easily). Some people manage this, but often universities balk at it because they require the freedom to publish, unfettered by the constraints of a funding source. Unfortunately, sometimes the sexy problems are the classified ones.

    (2) If you want to be the best at tackling a particular problem, sometimes it doesn't matter how much funding you get, because you still can't compete with industry. This is particularly applicable in hot, fast-paced fields where industry is the leader. In this case, you can sometimes make a name for yourself by carving a niche, but this is difficult to do, and often unsuccessful, and I would never try it as an untenured professor.

    Just my first thoughts.

    ReplyDelete
    Replies
    1. Good points. I don't know of any computer scientists in universities working on classified problems. I am sure they exist, but my guess is that most of that kind of work happens in places like national labs. At one point the DoD wanted to classify all of the work that had been done to date by the sensor networking community, and the various faculty involved pushed back hard. Some of them dropped DARPA funding rather than allow their work to be classified.

      It is very true that industry is far, far ahead of academia in certain areas, especially in large-scale computer systems. It used to be the case that a reasonably well-funded university group could compete with industry for large systems research. This is no longer true: the scales are orders of magnitude off.

      Delete
  6. I think the sweet spot lies at industry research institutions that have very broad research mandates, almost-unlimited funding, and low commercial obligations. The only example I can think of right now in the field of computer science is Microsoft Research. (No affiliation other than a fervent desire to be working there. If I sound like a fanboy, just look at the number of papers in top CS publication venues that come from MSR.)

    I think they are the Xerox PARC and Bell Labs of this generation, given that there is negligible or non-existent pressure for the work done there to have direct commercial benefits. It's unfortunate that with the software field being as huge as it is now, there is only one research institution that is as forward-looking and freely-funded as MSR.

    PS - As somebody whose MS thesis focused on peer-to-peer systems, I totally empathize with your perspective at the time. I am amazed at how little of the research from the 2001 - 2006 years seems to be used in "cloud" systems of today. I think the key dichotomy is that P2P research assumed unreliable, decentralized systems, but current cloud infrastructure is based on semi-reliable, centralized-though-redundant distributed systems.

    ReplyDelete
    Replies
    1. Indeed. I have many friends who work at MSR and I respect the hell out of the place. On the other hand, history has shown that running an academic research organization inside of a company is not generally a sustainable model (witness Xerox PARC, Bell Labs, etc.). MSR is clearly having lots of impact in the academic sphere -- the question is how much impact they are having on Microsoft's products.

      Google takes a fairly different approach; I recommend the following article for some background:

      http://cacm.acm.org/magazines/2012/7/151226-googles-hybrid-approach-to-research/fulltext

      Delete
    2. I've read that article, and it certainly is interesting. A sharp focus on producing immediate practically useful results is very good for the company and for its users.

      In that sense, MSR is somewhat ineffective. (Although, it is argued that more of their results do make it into their products than is readily apparent, such as programming language and static analysis research improving their developer tools, or distributed systems work shoring up Azure.) However, several MSFT stockholders -- including many MS employees -- look at the money put into MSR as an inefficient use of funds because so little of it directly impacts the bottom line.

      However, I think that is a rather short-sighted view, one that I myself once held. There is a place in industry for well-funded blue-sky research with few or no commercial obligations simply because, in the bigger picture, many ground-breaking advances don't necessarily have their origins in current product-based or other practical requirements. I'd say this is the role that MSR was always intended to fulfill, and I appreciate that Microsoft is taking the long-term view here rather than giving in to stockholder demands.

      Delete
    3. I agree that the long view is great, but don't forget that Microsoft is a publicly traded company (as is Google). Shareholders own it.

      Delete
    4. I agree, and honestly, MSFT as a business has not been doing that great in the last few years. The longer that trend continues, the more dire and pessimistic the future of MSR as an "independent" research institution. Many believe it will sooner or later become like the IBM and HP labs of today, i.e. much more application- and client-driven than foundational research.

      Delete
  7. I don't think anyone expects to have much academic freedom pre-tenure. As for post-tenure, I think you have a great deal of freedom - as you have observed in a comment above, some tenured faculty do "say fuck it."

    A final point about impact: as a tenured faculty member, you can take the long view. You can do the research which you think will have impact in the long term and wholly ignore whether you are publishing in top-tier conferences or not. That sounds pretty satisfying to me. As for funding and so forth, that depends a lot on your area - in some areas, lack of funding is not an impediment to research.

    Finally, your comment,

    ``I definitely know some tenured faculty who are coasting and care a lot less about where and how much they publish, or who tend to dabble rather than take a more focused research agenda post-tenure. Certainly you cannot get fired if you are not publishing or bringing in the research dollars anymore, but to me this sounds like an unsatisfying career. ''

    ...strikes me as off. This sounds to *you* like an unsatisfying career, but how does it feel to *them*?

    ReplyDelete
    Replies
    1. I don't know. As I said, "*to me* this sounds like an unsatisfying career." I'm not interested in coasting.

      Delete
  8. Matt this does seem to be the reality of life, analogous to "one persons freedom ends where another persons freedom begins". I don't think anyone expects full academic freedom regardless consideration of their own/other academic institutions about their research... nevertheless nice post on what are the main factors affecting your freedom in academia!

    ReplyDelete
  9. "Certainly you cannot get fired if you are not publishing or bringing in the research dollars anymore"

    Nope not true. My buddy, a tenure full professor at Kanas State (who publishes and brings in grant money) is being fired for poor attendance. Details here:
    http://barfblog.com/2013/03/bye-bye-bites-fired-for-bad-attendance/

    ReplyDelete
    Replies
    1. You can also get fired for faking research results or having sex with your students. What I said was that you cannot get fired for not publishing or bringing in research dollars. Your colleague was fired for not being present on campus, which is a very different thing.

      Delete
  10. Just to air a contrary view, I've always worked on pretty much what I wanted -- as a grad student, as an assistant professor, and as an associate professor. And if I think someone else is going to solve a problem, I let them have it, and go maximize my value by doing something that only I can do.

    Obviously I've been lucky with funding and students. More often than not, I've been able to convince people to go along with my ideas, perhaps because I generate a lot of ideas and then try to pick the ones that are most worth my time. I can enthusiastically argue for why they're cool and important.

    But I think my biggest piece of luck is being in a subfield that isn't controlled by "30-some-odd people." In NLP or ML, all of the decent researchers (including some senior grad students) are asked to review for all of the conferences. Each paper gets 3 double-blind reviews, and each reviewer only reviews 3-6 papers (depending on the conference), so they can do a careful job. The area chairs and program chairs also tend to be different from year to year, and they place a high value on originality and diversity.

    (Example: ACL 2012 had 2 program chairs, 29 area chairs, and 665 reviewers, who reviewed 571 long paper submissions (19% accepted) and 369 short paper submissions (20% accepted).)

    My one concession has been that I haven't put all my eggs in one high-risk, long-term basket, which might actually be the best use of my time. But in fact I like working on several things at once and it's wise in some ways. The interactions are stimulating and it gives my grad students a lot of choice.

    ReplyDelete
    Replies
    1. It is true that your interests can often naturally align with what the community deems to be publishable and fundable, but what would happen if you wanted to veer off into a totally crazy new area (e.g., automated translation of Klingon)? Would you be able to do it?

      Delete
    2. Well, I'd start by doing unfunded research in that area. The dean does pay 9 months/year of my salary with complete academic freedom, after all. Can I do the research all by myself? No, but at my department, there are plenty of undergrads and master's students around who are eager to get involved in research, as well as Ph.D. students who need to find qualifying projects. And some of my own Ph.D. students have external fellowships. If a bit of money is needed, e.g. to pay students over the summer, one typically has some discretionary funds (beginning with startup) or can get small amounts of exploratory funding from NSF or internal grants.

      The point of this early research is to show that the area is both interesting and feasible, and to drum up interest. Then I can go write a grant proposal if those first couple of papers weren't enough to scratch my itch.

      You're right that it can be hard to establish your credibility in a "new area." I have unsuccessfully proposed work to NSF in areas where I didn't yet have a track record. But the reviewers are entitled to be skeptical about a newcomer's chances of success. If I took the hint and teamed up on my proposal with someone who'd been around that block a few times, it wouldn't necessarily be a bad thing. The first step would be selling such a collaborator on my idea, one on one.

      If you meant "totally crazy" literally, well, a totally crazy area probably shouldn't be funded! You do have to justify that the work is worthwhile.[*]

      The question is whether the funders and program committees are systematically blocking GOOD original work in order to focus on hot areas, as you say. That would be bad. As I said, I'm glad to be in a subfield where that seems to happen less, in my humble opinion. YMMV.

      (To play devil's advocate, though, there is something to be said for a scientific community having some areas of focus at any one time. A lot of people get interested in topic X and work out all of the natural problems. There's a lot of useful discussion, through publications and in person, among folks who are thinking and learning about the same things at the same time. If topic X was worthwhile, this can leave a lasting impression on the field.)

      --------

      [*] Your example seems ill-chosen, since Klingon poses no special computer science problems that would motivate such a research program. On NSF's criteria, it ranks low on intellectual merit as well as broader impacts. The point of statistical machine translation is that it learns from the data: why limit to one language? If you train your general-purpose MT system on Klingon-English data, it should learn to translate Klingon into English. In fact one of my colleagues did include Klingon alongside many other languages in his machine translation project. Here's NY Times coverage from 2004: http://www.nytimes.com/2003/07/31/technology/what-s-next-from-uzbek-to-klingon-the-machine-cracks-the-code.html

      Delete
  11. I think there is a difference between freedom (i.e., the independence to do what you want) and whether you will get automatic and unequivocal support for doing anything. For example, I am "free" to walk in the park in a gorilla costume while singing opera, but I wouldn't necessarily expect accolades from others (let alone financial support) for doing so. Academic freedom is the freedom to try.

    How much freedom that is practically speaking is something I think each person views through the lens of their own experience. I think you probably chose industry for the right reasons and I think, also for right reasons, there are quite a few people who choose to be in, and love, academia. Its not like there is a decline in totally great people seeking academic jobs after all :-)

    In my own experience, I've felt tremendously free in academia (pre-tenure and otherwise). I don't think there is any company that would employ me to work on the range of topics I've covered in academia (some successful, some complete failures). Industry employers, by necessity, has goals and their labor force ultimately must reflect those goals. Academia provides an opportunity (for those who like it) to experiment with lots of different goals... and sometimes not even knowing what the goal is. Does that experimentation come with strings and costs? Absolutely. You need to sell your ideas to others for the purposes of publication, for funding, for convincing graduate students to join and so on. However, I think that skill is a requirement for pushing innovation broadly writ. Its not always enough to do something interesting and hope that "if you build it, they will come", but frequently you need to explain to people why its interesting and provide strong enough evidence that you can convince them. Learning how to do this better is part of what the Ph.D. process is really good at (when it succeeds).

    Academia, like all jobs, can be tough. Some people are better suited for it than others, just as with industry. For those who are well-suited to academia though, I think the freedom is absolutely real and a huge perk. Indeed, that and the excitement of working with students is what keeps me here year after year.

    ReplyDelete
    Replies
    1. Stefan, good points. You are one of those people I put in the category of being able to define new fields and have clearly been incredibly successful pursuing your own path and getting others to follow along with your vision (and to build a small empire doing it :-)

      We are in agreement that "experimentation comes with strings and costs". All I'm trying to do is combat what I believe is a naive view that many PhD students that I speak to seem to have, which is that being an academic means complete and total freedom with no strings attached.

      Delete
  12. Being a professor is not for everyone. It is a lot like launching a start up company. You must choose an important problem where there is not too much competition, you must known your own strengths and limitations, you must raise money, and you must recruit good staff. It is a tremendous amount of work. The big advantage is that you have tremendous freedom to follow the most interesting ideas where ever they lead, whereas at a company you get into trouble if you wander too far from the primary business of the firm. In my career, I have worked on a wide range of problems (from drug design to bird migration) and a wide range of techniques (from classification to anomaly detection to probabilistic modeling).

    As several other people have pointed out, your complaints about the challenges of publication are pretty pitiful. When I started out in graduate school, there was no such field as Machine Learning. (In fact, my MS advisor, Ryszard Michalski, named the field.) There were essentially no publication venues. ACM was hostile to anything related to AI, and even the AI Journal was skeptical of machine learning. So a group of us started the Machine Learning journal, and later the Journal of Machine Learning Research. Not only did this provide us a venue for publication, but it required us to take responsibility for deciding what constituted good research. This had a very salutary effect on the field.

    Your post treats the funding agencies and their priorities as immutable. But in fact, there are many ways for researchers to influence the priorities and directions of the agencies. You can talk to program officers and sell your ideas. Almost every agency has a way to give small amounts of seed money to promising ideas, and that can provide a good way to fund pilot work for a full proposal. You can write white papers, run workshops, serve on grant review panels, and even do a tour of duty as a program officer yourself.

    The key is that you must be entrepreneurial and take the initiative. If you aren't interested in doing that, then you should go work for someone who is.

    ReplyDelete
    Replies
    1. One thing that struck me wrong about this post is this sentence:

      "Now, there are some folks ... who are able to essentially start new fields and get the community to go along with them. I argue that most professors are not able to do this."

      I'd argue that if you're a professor at a top research institution, it's your *job* to try to do this.

      I really like Tom's example of how this happened in machine learning (perhaps because I'm a machine learning researcher).

      Phil Agre's article on "Networking on the Network" (not sure if it's still possible to find this) also provides a good example of how fields start up. It is, I am told, about finding a group with common interests and building consensus.

      Delete
    2. I absolutely agree that the most successful academics -- those that have managed to build up new fields and get the community to go along with them -- are the most "entrepreneurial" in the sense you mean (not in the sense of starting companies necessarily). Not everyone is good enough to do this.

      When I was just starting out as a junior faculty member, I was hanging out with Amin Vahdat (who is also at Google, by the way) and he gave me a piece of advice about my research -- he asked whether I would be able to turn what I was doing into a "franchise". This was great advice. At the time I was mostly working on problems that I personally cared about but that wouldn't have necessarily grown into a larger agenda. The point was that I was certainly "free" to work on those problems, but in doing so I was running a risk of not being able to get funding or to publish very much.

      Delete
  13. In my opinion, national labs strike a good balance between industrial research lab and academia.

    There is funding pressure in national labs as well, but DOE grant rates are slightly better than NSF. DOE is also somewhat more mission oriented, so you end up producing something worthy -- and not just one off paper, as they often tag academicians. You can work on multiple topics like in academia, but not all of it need to succeed or align with lab's long-term vision. That way it's probably may have more flexibility compared to some of the industrial research labs. As far as infrastructure goes, national labs may not have what Google hosts, but at least they are not off by an order of magnitude, better than University setting in many cases.

    Also, many universities are looking for tenure-track faculty with successful grant writing experience, national labs provide exposure to that. This is something you may not add on your CV while working for a company, say IBM, Google etc.

    Finally, you "own" your work in national labs. I am not sure if a fresh PhD student working in Google can publish a paper and "own" it as well. That paper is going to have some BIG names, e.g. M Welsh, S Ghemawat, J Dean. So, I am not sure if faculty at University perceive that paper as his own, though he may very well be the first author. I am not saying that professors are right in judging that person that way, just because the author list ends with name. May be that is a problem only if you are trying to move to academia from Industry. As Phillip Guo at Google recently noted "more than an year in Industry and your CV may become stale for academic jobs". Also, not many fresh PhD students get publish at Google (by my counts).

    Of course, these labs come with their own problems, like any other government agency. Politics, bureaucracy and what not -- may be Google is free of these evils, but academia is plagued with this
    too, what place is not?

    May be you have something to refute or add?

    ReplyDelete
    Replies
    1. I don't know much about the national labs so I can't comment there.

      I guess I don't understand this fetishism that the academic community has with publishing papers and author ordering and all of that. Far too much attention is paid to this in my opinion. We should be focused on having impact and improving the human condition, not whether somebody got a paper into some elite venue and whether they were the second or third author.

      Delete
  14. I am quite surprise that Matt's perspective spark a quite a discussion there. I definitely can relate to Matt's insight about "The other side of academic freedom". I used to work for a professor to establish a new research field which we thought would be an obvious for researchers from academia or industry. However, the truth is it is very difficult to convince people that the problem is worth to be considered seriously.

    Drawing from my own experience as a PhD student in CS, it is true many young faculties and graduating PhD students are adviced not chose to work on problems outside of the mainstream for exactly what Matt describes. However, there are some degree of academic freedom with a cost. My advisor and I did work on project outside the mainstream topic as our "PET" projects but not as the topic of my dissertation. The reason is I need the publications and funding for my study.

    It is very true there are many faculties, especially those well established professors, who are blessed with complete academic freedom and get to do whatever they like. But what Matt has shared in his blog is also true.

    H

    ReplyDelete
  15. I went into grad school and research because I wanted to work on something that could "change the world" in some small but meaningful way. Small, because it's just the work of one person, but meaningful because a timely good idea is potentially very powerful.

    From that perspective, I'm essentially only interested in research that can be meaningfully differentiated from other people's work. The value of contributing to basic research exhibits diminishing returns if you're just following the bandwagon. This was what I wanted to do as a student (although I only gradually realized that over time), and this is still what I want to do now.

    I work in machine learning. As others have mentioned, we don't have this problem of only accepting papers in "hot" areas. My impression is that the threshold for acceptance at a typical machine learning conference is significantly lower than that for a typical systems conference. For instance, ICML this year had hundreds PC members and accepted almost 300 papers. Of course, many papers will be forgettable (i.e., higher rate of false positives than a systems conference), but one should expect there to be more interesting papers (lower amount of false negatives). So in some sense, which papers and ideas rise to the top will be determined through a more democratic process.

    ReplyDelete
  16. Have just encountered your page and I guess you should be complimented for this piece. More power to you!

    ReplyDelete

Startup Life: Three Months In

I've posted a story to Medium on what it's been like to work at a startup, after years at Google. Check it out here.