My team at Google is wrapping up an effort to rewrite a large production system (almost) entirely in Go. I say "almost" because one component of the system -- a library for transcoding between image formats -- works perfectly well in C++, so we decided to leave it as-is. But the rest of the system is 100% Go, not just wrappers to existing modules in C++ or another language. It's been a fun experience and I thought I'd share some lessons learned.
Why rewrite?
The first question we must answer is why we considered a rewrite in the first place. When we started this project, we adopted an existing C++ based system, which had been developed over the course of a couple of years by two of our sister teams at Google. It's a good system and does its job remarkably well. However, it has been used in several different projects with vastly different goals, leading to a nontrivial accretion of cruft. Over time, it became apparent that for us to continue to innovate rapidly would be extremely challenging on this large, shared codebase. This is not a ding to the original developers -- it is just a fact that when certain design decisions become ossified, it becomes more difficult to rethink them, especially when multiple teams are sharing the code.
Before doing the rewrite, we realized we needed only a small subset of the functionality of the original system -- perhaps 20% (or less) of what the other projects were doing with it. We were also looking at making some radical changes to its core logic, and wanted to experiment with new features in a way that would not impact the velocity of our team or the others using the code. Finally, the cognitive burden associated with making changes to any large, shared codebase is unbearable -- almost any change required touching lots of code that the developer did not fully understand, and updating test cases with unclear consequences for the other users of the code.
So, we decided to fork off and do a from-scratch rewrite. The bet we made was that taking an initial productivity hit during the initial rewrite would pay off in droves when we were able to add more features over time. It has also given us an opportunity to rethink some of the core design decisions of our system, which has been extremely valuable for improving our own understanding of its workings.
Why Go?
I'll admit that at first I was highly skeptical of using Go. This production system sits directly on the serving path between users and their content, so it has to be fast. It also has to handle a large query volume, so CPU and memory efficiency are key. Go's reliance on garbage collection gave me pause (pun intended ... har har har), given how much pain Java developers go through to manage their memory footprint. Also, I was not sure how well Go would be supported for the kind of development we wanted to do inside of Google. Our system has lots of dependencies, and the last thing I wanted was to have to reinvent lots of libraries in Go that we already had in C++. Finally, there was also simply the fear of the unknown.
My whole attitude changed when Michael Piatek (one of the star engineers in the group) sent me an initial cut at the core system rewrite in Go, the result of less than a week's work. Unlike the original C++ based system, I could actually read the code, even though I didn't know Go (yet). The #1 benefit we get from Go is the lightweight concurrency provided by goroutines. Instead of a messy chain of dozens of asynchronous callbacks spread over tens of source files, the core logic of the system fits in a couple hundred lines of code, all in the same file. You just read it from top to bottom, and it makes sense.
Michael also made the observation that Go is a language designed for writing Web-based services. Its standard libraries provide all of the machinery you need for serving HTTP, processing URLs, dealing with sockets, doing crypto, processing dates and timestamps, doing compression. Unlike, say, Python, Go is a compiled language and therefore very fast. Go's modular design makes for beautiful decomposition of code across modules, with clear explicit dependencies between them. Its incremental compilation approach makes builds lightning fast. Automatic memory management means you never have to worry about freeing memory (although the usual caveats with a GC-based language apply).
Being terse
Syntactically, Go is very succinct. Indeed, the Go style guidelines encourage you to write code as tersely as possible. At first this drove me up the wall, since I was used to using long descriptive variable names and spreading expressions over as many lines as possible. But now I appreciate the terse coding approach, as it makes reading and understanding the code later much, much easier.
Personally, I really like coding in Go. I can get to the point without having to write a bunch of boilerplate just to make the compiler happy. Unlike C++, I don't have to split the logic of my code across header files and .cc files. Unlike Java, you don't have to write anything that the compiler can infer, including the types of variables. Go feels a lot like coding in a lean scripting language, like Python, but you get type safety for free.
Our Go-based rewrite is 121 Go source files totaling about 21K lines of code (including comments). Compare that to the original system, which was 1400 C++ source files with 460K lines of code. (Remember what I said about the new system implementing a small subset of the new system's functionality, though I do feel that the code size reduction is disproportionate to the functionality reduction.)
What about ramp-up time?
Learning Go is easy coming from a C-like language background. There are no real surprises in the language; it pretty much makes sense. The standard libraries are very well documented, and there are plenty of online tutorials. None of the engineers on the team have taken very long at all to come up to speed in the language; heck, even one of our interns picked it up in a couple of days.
Overall, the rewrite has taken about 5 months and is already running in production. We have also implemented 3 or 4 major new features that would have taken much longer to implement in the original C++ based system, for the reasons described above. I estimate that our team's productivity has been improved by at least a factor of ten by moving to the new codebase, and by using Go.
Why not Go?
There are a few things about Go that I'm not super happy about, and that tend to bite me from time to time.
First, you need to "know" whether the variable you are dealing with is an interface or a struct. Structs can implement interfaces, of course, so in general you tend to treat these as the same thing. But when you're dealing with a struct, you might be passing by reference, in which the type is *myStruct, or you might be passing by value, in which the type is just myStruct. If, on the other hand, the thing you're dealing with is "just" an interface, you never have a pointer to it -- an interface is a pointer in some sense. It can get confusing when you're looking at code that is passing things around without the * to remember that it might actually "be a pointer" if it's an interface rather than a struct.
Go's type inference makes for lean code, but requires you to dig a little to figure out what the type of a given variable is if it's not explicit. So given code like:
Finally, Go's liberal use of interfaces allows a struct to implement an interface "by accident". You never have to explicitly declare that a given struct implements a particular interface, although it's good coding style to mention this in the comments. The problem with this is that it can be difficult to tell when you are reading a given segment of code whether the developer intended for their struct to implement the interface that they appear to be projecting onto it. Also, if you want to refactor an interface, you have to go find all of its (undeclared) implementations more or less by hand.
Most of all I find coding in Go really, really fun. This is a bad thing, since we all know that "real" programming is supposed to be a grueling, painful exercise of fighting with the compiler and tools. So programming in Go is making me soft. One day I'll find myself in the octagon ring with a bunch of sweaty, muscular C++ programmers bare-knuckling it out to the death, and I just know they're going to mop the floor with me. That's OK, until then I'll just keep on cuddling my stuffed gopher and running gofmt to auto-intent my code.
ObDisclaimer: Everything in this post is my personal opinion and does not represent the view of my employer.
 |
| Plus, the Go language has a cute mascot ... awwww! |
The first question we must answer is why we considered a rewrite in the first place. When we started this project, we adopted an existing C++ based system, which had been developed over the course of a couple of years by two of our sister teams at Google. It's a good system and does its job remarkably well. However, it has been used in several different projects with vastly different goals, leading to a nontrivial accretion of cruft. Over time, it became apparent that for us to continue to innovate rapidly would be extremely challenging on this large, shared codebase. This is not a ding to the original developers -- it is just a fact that when certain design decisions become ossified, it becomes more difficult to rethink them, especially when multiple teams are sharing the code.
Before doing the rewrite, we realized we needed only a small subset of the functionality of the original system -- perhaps 20% (or less) of what the other projects were doing with it. We were also looking at making some radical changes to its core logic, and wanted to experiment with new features in a way that would not impact the velocity of our team or the others using the code. Finally, the cognitive burden associated with making changes to any large, shared codebase is unbearable -- almost any change required touching lots of code that the developer did not fully understand, and updating test cases with unclear consequences for the other users of the code.
So, we decided to fork off and do a from-scratch rewrite. The bet we made was that taking an initial productivity hit during the initial rewrite would pay off in droves when we were able to add more features over time. It has also given us an opportunity to rethink some of the core design decisions of our system, which has been extremely valuable for improving our own understanding of its workings.
Why Go?
I'll admit that at first I was highly skeptical of using Go. This production system sits directly on the serving path between users and their content, so it has to be fast. It also has to handle a large query volume, so CPU and memory efficiency are key. Go's reliance on garbage collection gave me pause (pun intended ... har har har), given how much pain Java developers go through to manage their memory footprint. Also, I was not sure how well Go would be supported for the kind of development we wanted to do inside of Google. Our system has lots of dependencies, and the last thing I wanted was to have to reinvent lots of libraries in Go that we already had in C++. Finally, there was also simply the fear of the unknown.
My whole attitude changed when Michael Piatek (one of the star engineers in the group) sent me an initial cut at the core system rewrite in Go, the result of less than a week's work. Unlike the original C++ based system, I could actually read the code, even though I didn't know Go (yet). The #1 benefit we get from Go is the lightweight concurrency provided by goroutines. Instead of a messy chain of dozens of asynchronous callbacks spread over tens of source files, the core logic of the system fits in a couple hundred lines of code, all in the same file. You just read it from top to bottom, and it makes sense.
Michael also made the observation that Go is a language designed for writing Web-based services. Its standard libraries provide all of the machinery you need for serving HTTP, processing URLs, dealing with sockets, doing crypto, processing dates and timestamps, doing compression. Unlike, say, Python, Go is a compiled language and therefore very fast. Go's modular design makes for beautiful decomposition of code across modules, with clear explicit dependencies between them. Its incremental compilation approach makes builds lightning fast. Automatic memory management means you never have to worry about freeing memory (although the usual caveats with a GC-based language apply).
Being terse
Syntactically, Go is very succinct. Indeed, the Go style guidelines encourage you to write code as tersely as possible. At first this drove me up the wall, since I was used to using long descriptive variable names and spreading expressions over as many lines as possible. But now I appreciate the terse coding approach, as it makes reading and understanding the code later much, much easier.
Personally, I really like coding in Go. I can get to the point without having to write a bunch of boilerplate just to make the compiler happy. Unlike C++, I don't have to split the logic of my code across header files and .cc files. Unlike Java, you don't have to write anything that the compiler can infer, including the types of variables. Go feels a lot like coding in a lean scripting language, like Python, but you get type safety for free.
Our Go-based rewrite is 121 Go source files totaling about 21K lines of code (including comments). Compare that to the original system, which was 1400 C++ source files with 460K lines of code. (Remember what I said about the new system implementing a small subset of the new system's functionality, though I do feel that the code size reduction is disproportionate to the functionality reduction.)
What about ramp-up time?
Learning Go is easy coming from a C-like language background. There are no real surprises in the language; it pretty much makes sense. The standard libraries are very well documented, and there are plenty of online tutorials. None of the engineers on the team have taken very long at all to come up to speed in the language; heck, even one of our interns picked it up in a couple of days.
Overall, the rewrite has taken about 5 months and is already running in production. We have also implemented 3 or 4 major new features that would have taken much longer to implement in the original C++ based system, for the reasons described above. I estimate that our team's productivity has been improved by at least a factor of ten by moving to the new codebase, and by using Go.
Why not Go?
There are a few things about Go that I'm not super happy about, and that tend to bite me from time to time.
First, you need to "know" whether the variable you are dealing with is an interface or a struct. Structs can implement interfaces, of course, so in general you tend to treat these as the same thing. But when you're dealing with a struct, you might be passing by reference, in which the type is *myStruct, or you might be passing by value, in which the type is just myStruct. If, on the other hand, the thing you're dealing with is "just" an interface, you never have a pointer to it -- an interface is a pointer in some sense. It can get confusing when you're looking at code that is passing things around without the * to remember that it might actually "be a pointer" if it's an interface rather than a struct.
Go's type inference makes for lean code, but requires you to dig a little to figure out what the type of a given variable is if it's not explicit. So given code like:
foo, bar := someFunc(baz)You'd really like to know what foo and bar actually are, in case you want to add some new code to operate on them. If I could get out of the 1970s and use an editor other than vi, maybe I would get some help from an IDE in this regard, but I staunchly refuse to edit code with any tool that requires using a mouse.
Finally, Go's liberal use of interfaces allows a struct to implement an interface "by accident". You never have to explicitly declare that a given struct implements a particular interface, although it's good coding style to mention this in the comments. The problem with this is that it can be difficult to tell when you are reading a given segment of code whether the developer intended for their struct to implement the interface that they appear to be projecting onto it. Also, if you want to refactor an interface, you have to go find all of its (undeclared) implementations more or less by hand.
Most of all I find coding in Go really, really fun. This is a bad thing, since we all know that "real" programming is supposed to be a grueling, painful exercise of fighting with the compiler and tools. So programming in Go is making me soft. One day I'll find myself in the octagon ring with a bunch of sweaty, muscular C++ programmers bare-knuckling it out to the death, and I just know they're going to mop the floor with me. That's OK, until then I'll just keep on cuddling my stuffed gopher and running gofmt to auto-intent my code.
ObDisclaimer: Everything in this post is my personal opinion and does not represent the view of my employer.
Matt, have you tried the margo and gocode systems? I use GoSublime with Sublime Text, but am otherwise a Vim snob, and if I hadn't gotten used to using Sublime Text for Go, I believe from what I've read that I'd be using these. The plugins turn Sublime Text into something very close to an IDE, and I believe they offer essentially the same functionality for Vim. They provide autocompletion hints that indicate the types of variables and functions. This is super helpful to me when I'm trying to figure out the type of something that isn't explicitly declared.
ReplyDeleteGo tips for Vim users: http://0value.com/my-Go-centric-Vim-setup
DeleteNope, I haven't tried them - will check them out!
DeleteThere's also "vintage" mode for SublimeText2, which gives the user a lot of Vim keyboard shortcuts, including command, visual, and edit modes.
DeleteHow to install go and run it on a local machine?
ReplyDeleteIf you have OS X, try homebrew (http://brew.sh/). With brew all the updates are easy. The recent upgrade from 1.1.1 to 1.1.2 was released through brew almost the same day it came out.
Deletehttp://golang.org/doc/install
ReplyDeleteThere's an installer for OS X, an MSI package for Windows and tarballs for Linux (most distributions have Go in their repositories).
Back when you were at Berkeley you were
ReplyDeleteenthusiastic about writing systems code
in Java. I remember listening to you talk
about what a good idea it was during your
dissertation talk ...
I used to be more enthusiastic about Java for systems programming than I am now. In part this is because the language and runtime libraries have evolved to a point where the standard way of doing things in Java is incredibly complex and verbose. The joke goes that you need a FactoryFactoryFactoryFactory in order to get anything done. A lot of anti-patterns have emerged in the Java space, which is kind of sad. At its core, the Java language and runtime are really good. The runtime libraries have gotten way, way too bloated.
DeleteWe are doing the exact same kind of rewrite at our company and would have the exact same things to say. Except we decided - but not with complete accord - to use full variable names (no abbreviations), coming from .NET land. The problem is that we find ourselves still breaking this convention because of practicality sometimes.
ReplyDeleteThe interface type issue is something we're still trying to figure out... Effective Go suggests ending single-method interface names in "er" but says nothing for multiple method interfaces. Maybe we'll add an "I" at the end of the names?
How about chaining the method names?
DeleteIn Go's standard library you find interfaces like the 'io.ReadWriteCloser' [ http://golang.org/pkg/io/#ReadWriteCloser ].
Of course this makes only sense with smaller interfaces.
You're right; if each interface had one method and you layered no more than two, this works, but even for us, "AccountLoaderUpdater" was a little much. :)
DeleteHard to say what works for your team, and I may not understand the example, but that I might call that an accounts.Store, or somepkg.AccountStore if needed.
DeleteIt may help when reading code that concrete objects happen to almost always be handled via pointer. So it's *bufio.Writer, *bytes.Buffer, etc., but io.Reader, crypto.Hash, etc.
Hi,
ReplyDeleteAbout your problem with not knowing types of foo, bar in
foo, bar := someFunc(baz)
I use Vim plugin called vim-godef. It is sort of like ctags, but better. It lets you jump to definition within your codebase as well as Go source. So you are just 'gd' away from learning what your function takes and returns.
Yeah, I have a similar kind of vim plugin, but damned if I can remember to use it :-)
DeleteWe're about to do the same thing at Twiitch (my video game company), for our server-side message processing. Currently everything is written in pretty lean Java, but the performance benefits can directly translate into cheaper server costs at scale.
ReplyDeleteOne thing that's stopped me jumping straight in is debugging/IDE support.
I know you don't like using an editor with a mouse, each to their own, however our programmers, me included, still like a debugger.
I've used LiteIDE with GDB support, which works pretty well (interactive debugging, stack, variables all work), but I'm curious how your team approached debugging. With 21k lines, I'm sure you had bugs that weren't caught by a unit test, so did you just printf or did you do something more interactive?
Shane.
gdb works fine with Go. I'm also a big fan of the old "add printfs everywhere" approach. But I'm old-school.
Delete:) ok, fair enough.
Delete"Unlike, say, Python, Go is a compiled language and therefore very fast".
ReplyDeleteWith the current day JIT and runtime optimizations, apart from the constant price for startup a lot of traditionally interpreted programs come really close to natively compiled code, particularly for long running programs such as servers. It is interesting to hear one of the Go designers talk about what he considers as "native" code on MSDN's Channel 9: http://www.youtube.com/watch?v=on5DeUyWDqI&t=3m30s
You don't have to convince me of this; I spent a lot of time working on JIT compiler optimization for Java. But please check out, for example: http://ziutek.github.io/web_bench/
DeleteOld...
DeleteCheck:
http://www.techempower.com/benchmarks/#section=data-r6&l=e80
Best regards,
Dobrosław Żybort
I am kind of upset that Google is not taking up Scala in a big way. The compiled code is almost always as fast as Java. The type system is much much better than Go and is capable of inferring much more than Go, with very advanced support for generics. Scala code is always much more compact than Go (true for most functional languages). Most features in other languages are simply implemented as a library in Scala. In particular Go channels are available as an Akka actor library in Scala. However Go has nothing comparable to Scala's Futures and Finagle libraries for async I/O support.
ReplyDeleteIf Google were to invest more resources in Scala, it can make developers world wide much more happier.
I understand that Go might be an improvement over C++/Java. But it pales in comparison to all the programming language research over the past 3 decades that has produced languages like Haskell, ML and Scala.
Personally for me, Go might have been a good choice 20 years ago, but right now, I just don't see why I should be using it when Scala can do everything better.
Scala doesn't solve all the problems that Go was designed to solve. And it makes some of the problems worse.
DeleteFor background: http://talks.golang.org/2012/splash.article
Engineers at Google enjoy compiling and testing their code in less than an afternoon.
DeleteI have worked as a Scala developer for two years now and must say the slowness of the Scala compiler is a huge downside.
DeleteThe pauses when compiling even a few files are long enough to constantly distract you from the problem you are trying to solve. I (subjectively) never felt like I was much more productive in Scala than in Java.
Indeed, one of the most important design considerations for Go is fast compilation. In huge projects (like we have at Google), this matters.
DeleteI use Scala in the eclipse IDE and it launches projects in less than 3 seconds. This is because of incremental builds. Full builds are slower. The project size is around a couple of thousand lines of Scala+Java code. While this may not seem huge, Scala code is way more compact than Java. The Scala compiler itself is less than 100K lines of code.
DeleteWith incremental builds, I have never been bitten by the slow compilation issue.
Frankly, with Scala I have written 50-100 lines of code, equivalent to maybe 500 lines of Java code and it very often works on the first run! Because the type checker really is that good and catches 90% of your typical bugs, which makes you much more aware of the remaining 10% of the bugs. My code quality in Scala is super high, when compared to code in any other language, except may be Haskell. I have written thousands of lines of Scala code with barely a bug reported by anyone at all.
If you set up your build to do incremental compiles, I just don't see any reason to switch to something else. I have looked at Go, they only have a limited set of built in generics and a far less sophisticated GC than the JVM, no IDE support(with autocomplete, background compilation etc, errors highlighted as you type) and far weaker ecosystem compared to Scala/Java. I just don't understand what benefit I can derive from using Go.
> For background: http://talks.golang.org/2012/splash.article
DeleteI did not find anything specific about scala here
So just go on using Scala if you like it. Just to balance your opinion, I'm happy Google is not taking up Scala in a big way.
DeleteMy my name is Greg Sudderth, just thought I'd say it because Anonymous is no fun!
DeleteFrom my point of view as guy who learned to program C with "ed" and the nroff-ing compiler (imagine C source all on the 0 column with nroff codes in it), go is a huge advancement. Get it that we used to think of the C compiler as a big macro assembler...a quicker way to generate more assembly.
I'm not comparing go to the Haskell/ML/Scala, that's comparison from up, going down. I'm looking at go, from assembly, going up. I can't write an OS in Java, or ML, or whatever, why would you? Even a custom language-specific-as-an-OS (think Lisp Machines) in the 80's...nope. BUT, I can write an OS in go, I can do all my utility programming that I might use Java 1.4 for, and, its going to run at flank speed, without the demon that few could tame...memory mismanagement. Only old timers remember what it was like before Purify.
From a guy that only knows a modicum of Python, the main comparative factor I'd use is "what libs does it have" and Python is going to win for sure, because its a well-loved and 20 year old language. Let's see what happens in go.
Speed, threading, a lot less religion than C++ and the STL, and the huge and nasty include issue dealt with. Wow. I can also bind over to another language, and use (e.g.) a lib like nanomsg, great! I think they got the whitespace all wrong but I am a AT&T flavored guy, not BSD. Oh well :)
G.
> BUT, I can write an OS in go
DeleteGo needs a garbage collector. You can write an OS in Haskell/ML/Scala as easily as you can write one in Go.
http://stackoverflow.com/questions/6638080/is-there-os-written-in-haskell
> So just go on using Scala if you like it. Just to balance your opinion, I'm happy Google is not taking up Scala in a big way.
DeleteI explained why I think Scala is superior to Go. I did not simply say that "I like it". There would be a balance if you could explain why Go is superior to Scala. I see the compilation speed advantage, but it is largely negated if you have a properly set up development environment with incremental compiles.
Is Scala/functional programming too hard? Or is it simply that people haven't tried it out or too afraid of switching paradigms and want to take smaller steps, like support for closures etc, but no advanced type system or curried functions?
Oh my
Delete> My code quality in Scala is super high, when compared to code in any other language, except may be Haskell.
> I have written thousands of lines of Scala code with barely a bug reported by anyone at all.
So you write code without mistakes with super high quality, you probably the only one on the planet!
I worked 5 years as C# developer and 1 year as Scala developer. I'm currently working as Scala developer, Go is my hobby.
On my daily job build times are killing me. I hear "incremental builds" from the audience. Oh really? Don't you use git? We use git, and each feature/bug is worked on in separate branch. So guess what happen when you switch branch? Incremental build fails, and starts from scratch. This takes so much time, terrible.
Then what I like in Go is that you can read and understand the code of other developers. In scala every developer invents it's own DSL, so there are a lot of $> #> -> ##> - go guess what it means in this file. Implicits are great too, go figure out what's happening. Tuples are great, but "results.map(_._1._2)" sucks.
Scala is super powerful. But great power means great responsibility. It's too easy in scala to write the code that no one can understand without a lot of research. Scala is modern C++ - long compilation, a lot of power and very complicated. Oh, and they also inserted XML inside the language. Why? XML times are gone, JSON, YAML is leading. Why XML?
> So you write code without mistakes with super high quality, you probably the only one on the planet!
DeleteYes I do, but I am not the only one. Ask around forums of Haskell/Scala/ML and this is a norm, not the exception. If you write Java in Scala then yes, the bug rate will remain high. I just wrote a 60 line program just yesterday, (a fairly complex diff program) and it ran correctly on the 2nd attempt. The 1st attempt had a bug, because of a misunderstanding about how zip
works on a Scala Map.
There are haskell programmers who publish code to hackage, without even running them (just compile) and several heavily used Haskell libraries have gone years on end without a single bug report.
Comparison of Go and Scala
Deletehttps://news.ycombinator.com/item?id=6333955
Might we get to find out what service it was, in time?
ReplyDeleteAnd very interesting to see the two dev-tools links in the comments, BTW. That's a lot more than I expected for a relatively new toolchain and an old-school editor.
>Also, if you want to refactor an interface, you have to go find all of its (undeclared) implementations more or less by hand.
ReplyDeleteOr use gofmt -r: http://golang.org/cmd/gofmt/
I'm curious about the details of the trouble with refactoring interfaces, partly because I've never worked on a system as large as Matt's.
DeleteIf the compiler can statically determine something doesn't satisfy a necessary interface, it seems like it often tells you; you get "[type] doesn't implement [interface]" if you pass a string to io.Copy() or do similar stunts.
Was the problem here mainly that the compilation errors aren't a clean/usable way to find the implementors? Or is it mostly that some of the important type-interface relationships aren't checked until runtime (e.g., by a type assertion)? If the latter, is the issue one of "generic" types (collections, etc.) storing interface{} or is it something more app-specific?
As a stand-in for real "X implements Y" declarations, you can always add various cheap statements that cause a compile-time interface-satisfaction check if there weren't such statements already--different phrasings I was playing with last night: http://play.golang.org/p/lUZtDdP5ia
DeleteWhy not Erlang?
ReplyDeleteWe rewrote a Java system in Erlang. The code size was 10x smaller and more scalable than that written in Java.
Because Google's hiring practices favor young people who went to Ivy League universities, not competent engineers who have heard of erlang. Alas, corporate culture comes from the top, and at the top of Google are two very arrogant, very young people.
DeleteThe reality is, Go doesn't deliver what Erlang delivers (and neither does Scala), but the troves of fad-following go fanatics aren't competent enough to know this.
They think "its' got go routines, so it's concurrent!"
Ever since the dotcom bubble, when every kid who wanted to be rich decided to be a CS major (without regard to any real interest in programming) our profession has deteriorated, good engineering practice is scoffed at and the mediocre run, like lemmings, after the latest fad.
This is why, for instance, Rails was so popular and after it node.js. The latter, though is its own punishment as these mediocre programmers deserved to be trapped in callback hell shipping crappy code.
Go is not a bad language... but it's popularity is out of proportion to what it's offered that is new.
Good lord, the CEO of Google is 40 years old and the Chairman is 58. You'll have to judge for yourself whether they're "arrogant," but they're not young.
DeleteThe reality is that Go is an awesome systems programming language, and a lot of people don't like functional programming (even a lot of people who aren't "kid[s] who wanted to be rich without regard to any real interest in programming.") Scala, Erlang, etc. are great for some people, but the idea that mature, comptent people use Scala and Erlang and hate Go, is absurd.
BTW how do you know Scala and Erlang are not "fads"? Scala is only 10 years old and Erlang has only been publicly available for 15 years.
I use both Go and Erlang, depending on my needs. I prefer Erlang's design and I adore OTP, however I'll be the first to admit that learning to think in Erlang was significantly more challenging than learning to think in Go, the latter being very similar to almost every other imperative language I've already used. Additionally when it comes to resource usage Go is pretty damn amazing (using very little CPU and memory compared to a lot of other languages I've used, like Ruby and Java) whereas the Erlang VM tends to use quite a bit of both RAM.
DeleteAnon #1 re: "Google's hiring practices favor young people who went to Ivy League universities," I don't even know where to begin. Are you even aware of who the designers of Go are?
DeleteNo scheduler preemption in Go compared to Erlang though :(
DeleteThe lack of per-process heaps and garbage collection is also scaring me away from Go.
Preemption is coming to Go in tip. See https://codereview.appspot.com/10264044
DeleteWe have to be mindful of how much garbage accumulates in our programs and when the GC pauses happen. I don't think it's any worse than any other garbage collected language, but I don't have data to back that up. Suffice it to say we're not feeling any pain because of it.
ReplyDeleteHas there been any movement on adding linking to FORTRAN libraries for doing numerical things?
ReplyDeleteDude, if you learned ctags you would take care of Issue #2 about function definitions, and maybe even #1 :) Go vim!
ReplyDeleteSmall typo:
ReplyDelete< Remember what I said about the new system implementing a small subset of the new system's functionality
> Remember what I said about the new system implementing a small subset of the old system's functionality
So what is the performance of Go system comparing to C++ one?
ReplyDeleteHi Matt,
ReplyDeleteDo you have any pointer for healing mechanism of systems? It means that problem is reported and system heals by its own self.
Please provide any pointers if you have..!!
I am definitely a bigger fan of Go than of C++, but how much of the benefit would you have gotten if you had done the same rewrite and same change in scope but in C++ instead?
ReplyDeleteAlso, I think it's interesting that you say you need features that an IDE were built to help you with (type resolution), then say "but I won't because they require a mouse" and then say you need even more IDE features (finding types that conform to an interface - that's text editor rocket science but IDE bread and butter).
A good IDE doesn't require more mouse use than a good text editor and even if they did, you're missing out on features you'd like to have because of self-imposed constraints. Are you trying to be productive or to prove a point? Whatever inhibition you have, get over it and just use an IDE. There are plenty of vi and emacs bindings.
Or explain why and prove me wrong, of course.