May 4, 2014

GopherCon 2014 favorite talks, notes

Posted in Software at 19:39 by graham

My favorite talks at GopherCon 2014:

  • Peter Bourgon: Best Practices for Production Environments Soundcloud were an early Go adopter, and this talk is their distilled learnings from two years of Go: Repo structure, config, logging, testing, deployment, and lots more. The one talk you need if you’re starting (or running) a significant Go project, and you want to do it right.

  • Petar Maymounkov: The Go Circuit: Towards Elastic Computation with No Failures Stick with this one. It starts off quite academic, but gets fascinating very fast. He models whole companies as a distributed system (based on CSP), then builds a language-agnostic cluster programming library where the API is a filesystem The Circuit. One of the highlights of the conference for me was building a filesystem with Petar in the hallway.

  • John Graham-Cumming: A Channel Compendium John is the author of those great in-depth Cloudflare blog posts. Solid talk about Go channels. nil channels always block, so you can ‘disable’ a select clause by setting a channel to nil. Closed channels never block. Heartbeat is just time.Tick, timeout is time.After. Go programs are small sequential pieces joined by channels.

Those are the three talks I enjoyed most. Here are my general notes on the conference and a few of the other talks.

GopherCon was 700 people, an unexpectedly large turnout for a first conference and a recent language. Go is definitely “the language of servers”; lots of talks by sysadmins and ops people, lots of sponsors from infrastructure businesses (Digital Ocean, Ubuntu / juju, Iron.io, Apcera, etc).

Denver is a very enjoyable city. Great beer (craft breweries all over), a bike share scheme and many bike lanes, warm dry weather, friendly locals, great food. It put me in mind of Portland without the rain. I urge you to go if you get the chance.

Rob Pike’s keynote presented the history of Go, all the way from ‘hello world’ in B in 1972, through K&R C, ANSI C, to Go. Initial discussions about Go started in 2007, first draft spec 2008, open source Nov 2009. He talked about why he thinks the language is such a success. He dissected a Go ‘hello world’ program, token by token.

For those following the “Go has no debugger” saga, Rob said that modern operating systems make it very difficult to write a debugger, because the debug interface came before multi-threaded programming. He also said ptrace is not very good (not his exact words – swearing was involved). A deep learning there maybe.

Derek Collison talked about optimizations he’d done on a message queue (there were three message queue talks at GopherCon – Go is definitely the language of software infrastructure):

  • Be garbage collector friendly: Don’t allocate short-lived memory on the heap. Do as few allocations as possible. Strive for ‘zero allocation’ in critical sections of code. Converting string to []byte, and vice-versa, is an allocate and copy, so don’t do it. This advice came up in several other talks.
  • Use -race in your automated test suite.
  • A fast parser is a non-destructive parser (again, to avoid memory allocation, copy, and garbage collection). Fast parsers tend to look like giant switch / if statements. Lots of array matches.

Josh Bleecher-Synder talked about Embedded Go and Bluetooth Low Energy. The number of memory allocations is as important is the size, both influence the length of the garbage collection pause.

Bluetooth Low Energy sounds really interesting. It’s so low power it can run on a button battery, and your device can be ‘always on’. It’s cheap and soon to be ubiquitous. Most new smartphones support it. Josh recommended this book for more details.

He also talked about building “crash only” programs: Assume you can crash at any point. Check everything at startup, make it all sane. No cleanup, recover or shutdown code, it’s all done on startup because if the battery runs out, your shutdown code isn’t going to run anyway.

Rob Miller of Mozilla talked about Heka. The embed Lua so they can do dynamic loading of plugins. Go starts Lua via cgo. Lua is very fast, and has Lua Parsing Expression Generators which sound interesting. He said in many cases they could pretty much copy the BNF grammar from the RFC into Lua, which makes writing parsers easy. They use protocol buffers to interact between Lua and Go. He also said don’t convert between string and []byte (or vice-versa) because it’s an allocate, copy and free operation.

Ben Johnson talked about high performance databases. Query execution is what everyone cares about, so focus on that. Protocol Buffers is much faster than JSON (60 times, for him). Like several other speakers, he said in performance critical sections limit memory allocations and memory copies. One of my favorite parts of his talk was his “Hierarchy of SPEED”: Network I/O > Disk I/O > Memory Allocation > Mutexes > Memory Access. Worry about them in that order.

Victor Vieux talked about Docker. Sadly I missed much of his talk (because I was learning to build a filesystem!). He mentioned that Go test files run in alphabetical order. He also showed the mflag package, a replacement for Go’s flag package which does both long/short options (GNU style) and option grouping. It sounds like a better flag. He showed gocover.io which will run code coverage on any public Go project (in a Docker container of course).

Ron Evans and Adrian Zankich had a very cool talk about Go powered robots with gobot.io. They showed Go on an Arduino, a Beaglebone Black, a Sphero, and then an AR Drone. Highlight was Go running on an AR Drone, using Open CV to identify faces, flown via a PS3 controller. You really need to see the video to enjoy this one, I’ll try and link when it’s online.

Blake Mizerany ran an entertaining panel of core devs:

  • Rob Pike shared an office with Bjarne Stroustroup at Bell Labs.
  • Rob advocated programs that write programs. You can have generics, just generate the type-specific version from a template at build time. Interestingly Ward Cunningham elsewhere has also mentioned that programs-that-write-programs are underused.
  • Interface’s are pure behavioral typing, structural typing. It’s important that you don’t declare what interfaces you implement, because it’s up to the user of you code to decide what interfaces they depend on.

Baron Schwartz, co-author of High Performance MySQL talked about the database/sql package:

  • sql.DB is not a connection, it manages a connection pool.
  • Call rows.Err() after your rows.Next() loop is done, that’s where you would get any errors that interrupted rows.Next.
  • Call rows.Close() as soon as you can, because a row set holds a database connection.
  • MySQL prepared statements are tied to a connection, whereas Go prepared statements are not. That means if between calling db.Prepare and using the prepared statement someone else took your connection, the statement may get prepared twice. He didn’t mention any problem with this, just a curiosity.
  • db.Ping() is how you open and validate a connection.
  • Because we’re in a connection pool, it makes no sense to issue connection-specific commands (SET NAMES UTF-8, USE <mydb>, etc). If you really need to bind a connection make a transaction: db.Begin() and db.Commit().
  • The database/sql package will auto-retry for you up to 10 times.
  • Make custom types by implementing Scanner and Valuer interfaces. Baron uses this to transparently encrypt sensitive information. Suggests gzip too.
  • If you really need database specific functionality, import the driver directly.

Alan Shreve talked about “Living outside the cloud”. He works on a command-line app which must run without network. Embed assets with go-bindata. Use termbox to abstract away curses.

Andrew Gerrand closed the conference:

Code is communication. Strive for good names, simple interfaces, precise docs. Don’t be too clever. Less is exponentially more.

Don’t build a thing, instead compose simple tasks.

Simplicity is hard. It takes time.

Leave a Comment

Note: Your comment will only appear on the site once I approve it manually. This can take a day or two. Thanks for taking the time to comment.