Skip to main content

David Shaw's Anton Supercomputer

Today, David Shaw of D. E. Shaw Research delivered a Distinguished Lecture in Computational Science here at Harvard (this is a new seminar series that Ros Reid and I cooked up to bring in a few high-profile speakers each semester). Of course, prior to forming D. E. Shaw Research, David founded D. E. Shaw and Co., a hedge fund which was one of the most successful quantitative analysis shops. Since 2001, David has been doing research full time -- D. E. Shaw Research develops both algorithms and customized machine architectures for molecular dynamic simulations, such as protein folding and macromolecule interactions. The result is the Anton supercomputer, a heavily customized machine with 512 specialized computing cores specifically designed for particle interaction simulations. It was a great talk and was very well attended -- I'll post the video once it's available.

David presented the algorithms and architecture behind the Anton machine. The goal is to run molecular dynamic simulations of molecules on the order of 50,000 atoms for 1 millsecond of simulated time. The performance target for Anton is 10,000 simulated nanoseconds for each day of compute time. To put this in perspective, the fastest codes on conventional parallel machines can muster around 100 ns of simulated time a day, meaning that 1 ms of simulated time would take more than 27 years to run. Anton can do the same in around 3 months. Prior to Anton, the longest simulations of these macromolecules that have been done to date are on the order of a few microseconds, which is not long enough to see some of the interesting structural changes that occur over longer time scales. (1 ms may not seem like a lot but it's amazing how much happens to these proteins during that time.)

Basically, each of the 512 nodes consists of a heavily pipelined special-purpose ASIC that is designed to compute the particle force interactions (using an algorithm called the NT method), along with a general-purpose processor that supports a limited instruction set. Communication is heavily optimized to reduce the amount of data exchanged between nodes. The processors are connected into a 3D toroidal hypercube and each processor "owns" a set of particles corresponding to a particular cube of space. They have built eight 512-node machines with a 1024-node machine coming online in March. They are working to make one of these available free to the research community to be hosted at the Pittsburgh Supercomputing Center.

The best part of the talk was the animations showing visualizations of a protein structure evolving over time. A movie showing just 230 usec of gpW showed substantial structural changes including partial unfoldings of the manifold. Apparently these dynamics have never been observed in other simulations and it's incredible how much insight the longer time scales can reveal.

David was very cool to talk with -- over lunch the conversation ran the gamut from computer architecture to quantum chromodynamics. I never got a sense of whether D.E. Shaw Research has any plans to commercialize this -- they really seem to be in it for the research value, and consider Anton just a tool to make discoveries. (Of course, I can imagine such a "tool" would be pretty lucrative to the drug-discovery industry.) This is a project is a great example of what's possible when computer scientists and domain scientists work closely together.

Comments

  1. I'm not a molecular dynamics expert, but folding@home claims to have done simulation at the millisecond timescale as well and for free (well, almost).
    http://folding.typepad.com/news/2010/01/major-new-result-from-foldinghome-simulation-of-the-millisecond-timescale.html

    Were there any comparisons to this done during his talk?

    ReplyDelete
  2. Anon - That is very interesting - here is the abstract for that paper: http://pubs.acs.org/doi/abs/10.1021/ja9090353. There seem to be some differences between this approach and Anton. Based on a quick read it seems that they ran several folding trajectories out to 40 usec but the aggregate time is 1.5 ms or so -- using some modeling to estimate the eventual state. Also, they seem to be using implicit solvent, rather than explicit (which is what Anton uses). Finally, the paper doesn't say how long it took to get the simulation results. I'd like to learn more.

    ReplyDelete
  3. Well, led me said this, my dear blogger, I think the algorithms is my entered life, and this blog is brilliant and very easy to read. Also all the links that you add here are interesting and have a lot to do with the principal matter in the text. Thanks for let me said it. Keep doing all this for us.

    ReplyDelete
  4. I should say it is a detailed article. Talks about a variety of things - something which I never thought could exist. What I found different in your article is the way you have gone about to explain the topic in a simplistic way.

    ReplyDelete

Post a Comment

Popular posts from this blog

Why I'm leaving Harvard

The word is out that I have decided to resign my tenured faculty job at Harvard to remain at Google. Obviously this will be a big change in my career, and one that I have spent a tremendous amount of time mulling over the last few months.

Rather than let rumors spread about the reasons for my move, I think I should be pretty direct in explaining my thinking here.

I should say first of all that I'm not leaving because of any problems with Harvard. On the contrary, I love Harvard, and will miss it a lot. The computer science faculty are absolutely top-notch, and the students are the best a professor could ever hope to work with. It is a fantastic environment, very supportive, and full of great people. They were crazy enough to give me tenure, and I feel no small pang of guilt for leaving now. I joined Harvard because it offered the opportunity to make a big impact on a great department at an important school, and I have no regrets about my decision to go there eight years ago. But m…

Rewriting a large production system in Go

My team at Google is wrapping up an effort to rewrite a large production system (almost) entirely in Go. I say "almost" because one component of the system -- a library for transcoding between image formats -- works perfectly well in C++, so we decided to leave it as-is. But the rest of the system is 100% Go, not just wrappers to existing modules in C++ or another language. It's been a fun experience and I thought I'd share some lessons learned.

Why rewrite?

The first question we must answer is why we considered a rewrite in the first place. When we started this project, we adopted an existing C++ based system, which had been developed over the course of a couple of years by two of our sister teams at Google. It's a good system and does its job remarkably well. However, it has been used in several different projects with vastly different goals, leading to a nontrivial accretion of cruft. Over time, it became apparent that for us to continue to innovate rapidly wo…

Running a software team at Google

I'm often asked what my job is like at Google since I left academia. I guess going from tenured professor to software engineer sounds like a big step down. Job titles aside, I'm much happier and more productive in my new role than I was in the 8 years at Harvard, though there are actually a lot of similarities between being a professor and running a software team.

I lead a team at Google's Seattle office which is responsible for a range of projects in the mobile web performance area (for more background on my team's work see my earlier blog post on the topic). One of our projects is the recently-announced data compression proxy support in Chrome Mobile. We also work on the PageSpeed suite of technologies, specifically focusing on mobile web optimization, as well as a bunch of other cool stuff that I can't talk about just yet.

My official job title is just "software engineer," which is the most common (and coveted) role at Google. (I say "coveted&quo…