February 29, 2012

Go after four months #golang

Posted in Software at 03:56 by graham

I’ve been using the Go programming language for over four months in my spare time, mainly to write an IRC client (hatcog) – here is my trip report.

Go sits somewhere between C and Python. It has the static type checking and bit-twiddling powers of C (and pointers!), yet much of the speed of development and conciseness of Python (e.g. string split and join). Bruce Eckel called it a language designed to create servers, and I agree wholeheartedly.

Go is a C family language, both in the sense that it looks like C, and in the sense that you can easily call C libraries. Here is hello world:

package main;

import "fmt"

func main() {
    fmt.Println("Hello World!");
}

Interlude: Cooking up Go

This is what I think the recipe for Go looked like:

  1. Go back to the 70s / 80s and find a bunch of great programmers. Say, Ken Thompson, Rob Pike, people like that. Marinate them in Bell Labs for 30 years, during which time they code in C, keep developing Unix, invent Plan 9, UTF-8, and other wonderful things.

  2. Take them out, seduce them with Python, wow them with Google-scale computing. Add more amazing programmers (Brad Fitzpatrick for example), stir in Google’s near-unlimited resources.

  3. Ask them how they would do C now, if they could start from scratch.

Observations

The C legacy is very strong in Go, and not just in the curly braces. For example object-orientation is done with a struct plus some methods.

Capitalization is significant. The visibility of a name outside a package is determined by whether its first character is upper case.

Multiple return values instead of exceptions. To signal an error occurred in your function you return the error, like in C, but you can return multiple values, like in Python. Error checking typically looks like this:

retval, err := myFunc(param)
if err != nil {
    // handle it
}

The style guide is whatever program gofmt says it is. It will often just fix it for you. Use whatever style you want, then run it through gofmt before checking-in.

Interfaces, with no explicit implementations. If you have the methods of that interface, you count. That gives statically checked duck typing. Lovely.

Go has arrays, but you usually use slices instead. A slice works like a Python slice, but internally it’s a smart pointer to an array, and has a different type (which matters because we’re doing static typing remember). The array is it’s backing storage. I found this tricky to understand at first.

Everything is passed by value: Objects (structs) and Arrays are copied before being passed. For arrays it doesn’t matter because you’re usually passing slices (which wrap pointers – as do maps). When defining your own types, you have to remember to always pass a pointer.

The compiler is strict: Unused variables are a compiler error.

Negatives

The one big potential negative is that it’s compiled. In Python I often find myself stepping into third party libraries. Always having the source code is fantastic. One print statement deep in a someone else’s code can really help comprehension. No being able to do this in Java was a problem. I don’t have enough experience with Go to say if this will be a problem during debugging or not.

A read-eval-print loop would have made learning Go much easier. I can’t overstate how often I try out simple commands, like date formatting, in ipython. In Java I used beanshell.

You can’t daemonize, because of the way go-routines work. This could be fixed soon, see Russ Cox’s message here. In practice I usually use Upstart, so it doesn’t cause a problem.

Go has an un-googlable name. The convention is to use golang as a search term.

The language is still very young and in flux. See for example the GitHub network for gocurse. v1 of Go is due in the first half of this year (2012), so hopefully things will settle down.

The extremely useful string manipulation methods are all in the strings package, instead of being methods on the string object. There are objects in many packages, so there is no doubt a very good reason for string not having these methods. That’s the thing about Go, there’s usually a very good reason for everything it does. Learning Go is an education. Oh, that’s a positive.

Positives

Old programs read like quiet conversations between a well-spoken research worker and a well-studied mechanical colleague, not as a debate with a compiler – Dick Gabriel.

Russ Cox uses that Dick Gabriel quote to illustrate what they are aiming for with Go. In some hard-to-define way, Go feels serious and grown up. It has high expectations of you.

A familiar language. As a Python programmer with a Java and C background, I was productive very fast.

A modern language. Go has everything you’d expect a modern language to have: Unicode, garbage collection, multi-core (GOMAXPROCS), built-in map, string and array types, closures, unit-testing , an official style guide (‘gofmt’), reflection, etc

The documentation is excellent, and usually available from the command line: godoc <package>.

The compiler is very fast, fast enough that you can use Go as a scripting language. Use gorun as your hash-bang line.

Static typing makes programming easier. That is a debatable point, but it does for me. Static typing makes code easier to read, and make me feel a little more comfortable that things won’t go boom in the middle of the night.

Concurrency

The one big NEW (to me at least) thing Go does very well is concurrency. The keyword go starts a new go-routine. That’s what you use instead of sub-processes, threads, or co-routines. The go-routine might run on the same core / processor as your current one, or it might not. You don’t worry about that.

Go-routine’s communicate on channels. You epoll those channels using the select keyword.

Here is the heart of hatcog, my IRC client. A go-routine listens on the port connected to the server and puts incoming data on channel fromServer. Another go-routine listens for keyboard input, and puts that on channel fromUser. The middle bit watches both channels:

for self.isRunning {

    select {
    case serverData, ok = <-fromServer:
        if ok {
            self.onServer(serverData)
        } else {
            self.isRunning = false
        }

    case userInput, ok = <-fromUser:
        if ok {
            self.onUser(userInput)
        } else {
            self.isRunning = false
        }
    }

}

More details on Go’s concurrency model

Do you see how that makes servers so much easier? You could write a webapp that’s it’s own server. The “C10K problem” might not be a problem at all.

Conclusion

Would I be happy working with Go as my main language? Yes, I would. It’s a joy to work with, and I got productive very fast.

Am I using it instead of Python for all my new projects? No, I’m not. There are two reasons for that. Firstly, it’s a very young language, so library availability is limited (for example, I need curses). Second, well, Python is just so amazing.

If I was still doing Java, or (heaven forbid) C++, I would invest heavily in Go. It was designed to replace them, and it does that well.

As a Python guy, the summary is more nuanced. I see the benefit of static typing. Other claimed benefits of Go over Python are that it’s faster, and that it’s “better at scale”.

For some things I’ve done Go has been faster, for other things Python. The biggest difference is probably in how well I write the code. Although I love speed in principle, nothing I do is CPU bound.

The “better at scale” argument doesn’t really apply to what I do for a living, which is building webapps with Django. We scale performance-wise by adding servers, and code-wise by adding small self-contained ‘apps’ (in the Django sense).

Go’s sweet spot is building servers. It makes concurrency safer and easier than the current options. I’m going to keep using it for that. And because it’s fun.

Next you might want to page through this Go presentation, or dive in to the (rather academic) tutorial.

(Did I make a mistake? Please correct me in the comments. Thanks!)

32 Comments »

  1. cheeky said,

    March 17, 2014 at 16:00

    do you have any example codes for creating a simple web chat application using GO language. I badly needed it for our project due on Saturday. I do not have any idea how am I going to do that web chat client-server application. Thanks.

  2. Craig said,

    January 28, 2014 at 08:53

    @Andrew, the kernel and all the functionalities it provides are written in C and their design is completely guided by C idioms. C also provides the low level ABI of most other languages. Sure, you don’t need to know C to be able to write code, but you absolutely do need to know C to have a legitimate claim to any knowledge of systems/kernel/OS programming or any kind of knowledge of how the other languages you use actually work. The Go toolchain is written in C. The Go string representation is implemented in C, using a C idiom. etc. etc.

  3. Craig said,

    January 28, 2014 at 08:38

    “The one big potential negative is that it’s compiled. In Python I often find myself stepping into third party libraries. Always having the source code is fantastic”

    Surely that’s your fault for using closed source code, rather than the languages fault for allowing binary-only distribution (even though you can just as easily ship Python bytcode).

  4. Ralph Corderoy said,

    July 5, 2013 at 12:20

    Rob et al didn’t marinate at Bell Labs for thirty years coding in C. Between C and Go were they and their colleagues created and worked in languages like Newsqueak, Alef, and Limbo. All CSP-influenced, e.g. channels for synchronisation and communication.

  5. Visto nel Web – 76 | Ok, panico said,

    April 28, 2013 at 08:12

    [...] Go after four months #golang ::: Graham King [...]

  6. SeanVN said,

    December 2, 2012 at 02:50

    Sick. It has no desktop applications support in it at all. Nor is it really possible to use shared libraries to make up the difference. The shared library support that there is, is whacko-jacko. Alternatively you have to use a mixed programming approach using some mutant c/go hybrid. Generally the language is well designed but they pushed the self-destruct button around the shared library issue. Java tried to address the same issues by supplying GUI libraries and native classes, even there that was not a very good solution (but better than the solutions in go).

  7. Yarko said,

    September 3, 2012 at 17:37

    … ipython => goplay;

    As Bill W. mentioned: play.golang.org; or: tour.golang.org; or: build your own local goplay: (in the sources, misc/goplay – at least on mac w/ brew, it’s not build by default, but trivial to build / link for daily use)

    RE: Python or Go? See Bruce Eckel’s “Calling Go from Python via JSON-RPC (http://www.artima.com/weblogs/viewpost.jsp?thread=333589)

    There appears to be thrift (thrift.apache.org), also thrift4go (https://github.com/pomack/thrift4go), but I don’t know much about the state it’s in. It seems thrift for python is current, so having thrift for go would complete the interchange, it would seem.

  8. Bill Waskinski said,

    August 27, 2012 at 19:11

    For a read-eval-print loop, try the online Go Playground if you haven’t already. I’ve found it great for trying new stuff.

  9. ljgww said,

    August 23, 2012 at 17:58

    Thank you for this unbiased view of Go(lang). It helps to put things in perspective.

    On the other hand it depends what perspective is as one can encounter Go coming from different worlds.

    If one is coming from compilers Go is great (syntax) refreshment, yet if you are coming from scripting world you may be misled with apparent similarity of syntax and lack of functionality which can be frustrating.

    After some thought given to Go (from 30000ft) my impression is that we are again into waters of bearing Yet Another C Child by crippling original C concepts while introducing new concepts. (as we have seen all these language attempts happening in the past 2-3 decades which have been not always up to their promises)

    I expected more from Go considering the time it was born, but fully understand that it is a language with a purpose. If it fits that purpose, that is great, I wish it luck. But it is important that limiting the purpose is also limiting wider adoption.

    IMHO Go shall be evaluated from C perspective and not from Pythonian (or scripting perspective e.g. PERL, PHP or even Ruby) as if it is seen from C perspective it can be more than welcome innovation especially on the line of parallel or multicore programming (in this waters we currently have, pretty much, only C and Fortran, which is not enough).

    Also I believe it shall jump over mobile boundary much more. It could be native system language on mobile platforms and I fully understand that it is not stable, explored and widely adopted so that mobile OSes will adopt it. For the time being mac/lin/win cross platform ability is highly appreciated.

    (*) when I say “not stable” I am not refering to stability of compiler or bugs it is more of “conceptual” language stability, simillar to early Java development when lot of ideas have been dropped off, whole libraries changed. I hope that programming world learned a lot from Java case.

  10. gophi said,

    August 21, 2012 at 08:06

    “The language is still very young and in flux.”

    The language itself is no longer in flux since the Go 1 release.

  11. Andrew said,

    July 19, 2012 at 14:30

    I never cared to learn C and I”m very comfortable with GO. Knowledge of C is not a requirement to understand how computers work or how to code them.

  12. graham said,

    May 3, 2012 at 17:19

    @Adrian Thank you! I have updated the text.

  13. Adrian Ratnapala said,

    April 28, 2012 at 10:34

    I might be wrong here, but I do not think Thompson, Pike, et. al, invented Unicode. They did invent UTF-8. Or at least that’s what Pike says.

    http://www.cl.cam.ac.uk/~mgk25/ucs/utf-8-history.txt

  14. voidlogic said,

    March 23, 2012 at 07:00

    @uri “I would love to see a tutorial on moving from Python to Go (i.e. filling in the gaps that you need if you lack a C/Java background.) Any ideas?”

    It is going to take more than a tutorial. C/Java is a fundamental component of a computer science education. Without knowing C you will never understand Go (or Python for that matter) and the computer, especially at the low-level, will remain mysterious to you.

    Why is this important? It is not- unless you are going to write efficient or high performance or scalable applications. If you don’t know C/C++/Java then chances are you do not understand the details multi-threading (where details are everything), when to use and how to implement the “right” data structures or algorithmic complexity analysis.

    My advise, learn C and Go. Probably read an algorithms book too.

  15. graham said,

    March 20, 2012 at 21:28

    @mike rosset Thanks, that’s definitely the best choice. I didn’t include it because the “go” command isn’t in “release” yet, and rc1 wasn’t out.

  16. mike rosset said,

    March 19, 2012 at 22:18

    In addition to gorun. you can also use “go run main.go” which is a more official way of doing things.

    I also created a go program to register and run go extensions with binfmt and it uses go run, another option if you are looking to get around the shebang issue. But its a Linux only solution.

    https://github.com/str1ngs/go-binfmt

  17. graham said,

    March 13, 2012 at 06:31

    @uri: In my experience most Go material does seem targeted at system developers, and expects you to have a C background. I think this will change when Go 1 is out. There’s probably several books waiting to be published. One already out (and free) which helped me is Learning Go

  18. uri said,

    March 12, 2012 at 16:23

    I would love to see a tutorial on moving from Python to Go (i.e. filling in the gaps that you need if you lack a C/Java background.) Any ideas?

  19. Achilleas Margaritis said,

    March 10, 2012 at 19:33

    Go a replacement for C++? please.

  20. graham said,

    March 8, 2012 at 23:10

    @Josh Ball: Four months only! :-) You can use “interface{}”, as in make(map[string] interface{}). I haven’t had any problem so far.

  21. graham said,

    March 8, 2012 at 22:45

    @alb By “type-safety” I simply mean that the compiler says things like “You can’t add an int to a string, you fool!”. I’ve edited the post to say “static type checking” instead. Thanks.

  22. Josh Ball said,

    March 8, 2012 at 06:58

    In your four years of experience, how do you feel about the lack of polymorphic data types (other than arrays). Has it not come up as an issue for you? Do you have a standard work around?

  23. alb said,

    March 8, 2012 at 05:51

    “It has the type-safety and bit-twiddling powers of C…” The type safety of C? You might want to check that. C is not generally called type safe. Maybe you meant C++ or C#?

  24. graham said,

    March 7, 2012 at 20:16

    @Alexander I can’t give an example, so I’ve edited that sentence. Many months ago I did some tests with strings and hash maps, in which Python looked faster. I was a total beginner in Go, and the differences were small enough that it probably depended on what else happened on my machine during the run. Hence I have nothing to say about performance. Thanks for bringing it up.
    Edit: My test was very similar to this: https://groups.google.com/group/golang-nuts/browse_thread/thread/9e97baa897f95c7b?hl=fr&pli=1

  25. graham said,

    March 7, 2012 at 20:07

    @whoops I agree, gorun is great. I link it in the article.

  26. jessta said,

    March 7, 2012 at 02:12

    To respond to some of the negatives:

    “The one big potential negative is that it’s compiled.” The go tool encourages distribution in code form. It’s actually quite annoying to distribute a compiled version of a package. This is intentional.

    “A read-eval-print loop would have made learning Go much easier” There is a reasonable simulation of a REPL here: https://bitbucket.org/binet/igo Eventually someone will write a proper interpreter for Go.

    “You can’t daemonize, because of the way go-routines work” You can’t daemonize because of the way threads work. All threaded programs have problems with fork(). Of course, daemonizing is a short shell script.

    “Go has an un-googlable name.” ‘Go’ is a word used in a lot of contexts and it’s use as the name of a programming language is relatively new. If you give it context then you get the results you want. “go programming” gives the work context.

    “The extremely useful string manipulation methods are all in the strings package, instead of being methods on the string object.” None of the built in types have methods, this is intentional to avoid confusion in relation to interfaces.

  27. Steven said,

    March 6, 2012 at 23:22

    A few points:

    1. Go is much more type (and memory) safe than C ;)
    2. It is possible to daemonize. You could use a sync.WaitGroup to make main wait for any number of goroutines to exit before exiting. But more directly, if you add defer runtime.Goexit() to the top of your main function, your program will wait for all goroutines to finish before exiting.
    3. You can get pretty close to a REPL with goplay, but different… it just compiles so fast you don’t need to interpret it to make it seem interactive. I’m sure a tool for Go will come along at some point that combines all the benefits of a REPL with none of the drawbacks.
    4. Most Go code is distributed as source at the moment. That might change with wider comercial adoption. Remember that not everybody wants to open-source their code, so while it may be inconvenient to you if the don’t, that’s their choice, not the language’s.
  28. whoops said,

    March 6, 2012 at 18:28

    i recommend looking at gorun (https://wiki.ubuntu.com/gorun) to allow you to do #! scripting. it is a really great tool created by one of the best coders contributing to the go ecosystem. otherwise i agree, the compilation phase is a drag!

  29. Alexander Solovyov said,

    March 6, 2012 at 13:41

    At the moment, it’s not faster. In every personal experiment I’ve tried, Python has been faster.

    Sorry, can you give an example? At anything I’ve tried Go was faster than Python (I’m Python guy mostly, and I love Python). Mostly even faster than PyPy.

  30. Alex said,

    March 2, 2012 at 18:28

    Graham, this is a fantastic post comparing and contrasting go against other languages. Thanks for taking the time to share your experiences.

  31. graham said,

    February 29, 2012 at 22:45

    @Max Thank you, that’s a good point. It applies to slices too. I’ve updated the post.

  32. Max said,

    February 29, 2012 at 08:27

    “Everything is passed by value – yes everything, maps and arrays are copied before being passed. Hence you often use pointers.”

    Maps are indeed passed by value, but the internal structure of a map is not copied – so passing a map is efficient (and changes made by the caller will be visible by the callee).

    max

Leave a Comment

Note: Your comment will only appear on the site once I approve it manually. This can take a day or two. Thanks for taking the time to comment.