Welcome ! This is the personal site / blog of Graham King. Most people come for the credit card generator, but I think the Categories (top right) are more interesting.

November 10, 2014

Release It: Write software for production

Posted in Software at 06:12 by graham

We need to design software to run in production. That’s the main lesson of Michael T. Nygard’s Release It. We often think of shipping the system as the end of the project, when in practice it is just the start.

Release It is an enjoyable book with some excellent production war stories. It suffers from being a little too broad in concepts, and a little too narrow in examples (all enterprise J2EE webapps). Despite this I’d recommend spending some time with it because it advocates a very important and easily overlooked idea: don’t code to pass the QA tests, code to avoid the 3am support call.

What follows are my notes from the book, which are a mixture of what the book says and what I think, grouped in categories that make sense to me.

Read the rest of this entry »

November 4, 2014

What a visit from Reddit looks like

Posted in Society at 05:58 by graham

My credit card generator was recently on the front page of reddit. Here’s what that looks like.

Here’s htop during peak traffic (click for larger view):

100% htop

Read the rest of this entry »

August 23, 2014

Learning assembler on Linux

Posted in Software at 04:49 by graham

For entertainment, I’m learning assembler on Linux. Jotting down some things I learn here.

There are two syntaxes, AT&T and Intel (Go uses it’s own, because Plan 9). They look very different, but once you get over that the differences are minimal. Linux tradition is mostly AT&T syntax, MS Windows mostly Intel.

There’s no standardisation, so each assembler can do things it’s own way. as, the GNU Assembler is the most common one on Linux (and what gcc emits by default), but nasm, the Net wide Assembler is very popular too. Code written for as will not assemble in nasm.

Read the rest of this entry »

June 28, 2014

Dump Go Abstract Syntax Tree

Posted in Software at 20:09 by graham

Go has good support for examining and modifying Go source code. This is a huge help in writing refactoring and code analysis tools. The first step is usually to parse a source file into it’s Abstract Syntax Tree representation. Here’s a complete program to display the AST for a given Go file:

package main

import (
    "go/ast"
    "go/parser"
    "go/token"
    "os"
)

func main() {
    fset := new(token.FileSet)
    f, _ := parser.ParseFile(fset, os.Args[1], nil, 0)
    ast.Print(fset, f)
}

Use:

  • Save that as goast.go
  • Build it: go build goast.go
  • Run it: ./goast <myfile.go>

May 24, 2014

Sync, a Unix way

Posted in Software at 05:49 by graham

Ever since Dropbox, I’ve been searching for a self-hosted, secure (and now Condi-free) way of keeping my machines synchronised and backed up. There are lots. I tried many, wrote a couple myself, but none were exactly what I wanted.

My problem was thinking Windows, looking for a single program. Once I started thinking Unix, looking for modular components, the answers were obvious.

Storage

First we need a remote master storage to sync against, somewhere to backup our files. And we want that exposed as a local filesystem. I use the most obvious answer, sshfs:

sudo apt-get install sshfs
mkdir -p /home/graham/.backup/crypt  # Why 'crypt'? Read on.

sshfs server.example.com:backup /home/graham/.backup/crypt

You can use any storage that can appear as a filesytem, such as FTP (via curlftpfs), NTFS, and many others.

Encryption

There’s two kinds of data: public data, and encrypted data. We want the second kind. Just layer encfs:

Read the rest of this entry »

May 4, 2014

GopherCon 2014 favorite talks, notes

Posted in Software at 19:39 by graham

My favorite talks at GopherCon 2014:

  • Peter Bourgon: Best Practices for Production Environments Soundcloud were an early Go adopter, and this talk is their distilled learnings from two years of Go: Repo structure, config, logging, testing, deployment, and lots more. The one talk you need if you’re starting (or running) a significant Go project, and you want to do it right.

  • Petar Maymounkov: The Go Circuit: Towards Elastic Computation with No Failures Stick with this one. It starts off quite academic, but gets fascinating very fast. He models whole companies as a distributed system (based on CSP), then builds a language-agnostic cluster programming library where the API is a filesystem The Circuit. One of the highlights of the conference for me was building a filesystem with Petar in the hallway.

  • John Graham-Cumming: A Channel Compendium John is the author of those great in-depth Cloudflare blog posts. Solid talk about Go channels. nil channels always block, so you can ‘disable’ a select clause by setting a channel to nil. Closed channels never block. Heartbeat is just time.Tick, timeout is time.After. Go programs are small sequential pieces joined by channels.

Those are the three talks I enjoyed most. Here are my general notes on the conference and a few of the other talks. Read the rest of this entry »

March 2, 2014

Raw sockets in Go: IP layer

Posted in Software, Uncategorized at 00:41 by graham

In the Internet protocol suite we usually work at the transport layer, with TCP or UDP. Go (golang) has good support for working with lower layers. This post is about working one layer down, at the IP layer.

If you want to use protocols other than TCP or UDP, or craft your own packets, you need to connect at the IP layer.

Receive

Let’s read the first ICMP packet on localhost:

package main

import (
    "fmt"
    "net"
)

func main() {
    protocol := "icmp"
    netaddr, _ := net.ResolveIPAddr("ip4", "127.0.0.1")
    conn, _ := net.ListenIP("ip4:"+protocol, netaddr)

    buf := make([]byte, 1024)
    numRead, _, _ := conn.ReadFrom(buf)
    fmt.Printf("% X\n", buf[:numRead])
}

Read the rest of this entry »

March 1, 2014

Three best programming books

Posted in Software at 23:19 by graham

Here are my three favorite programming books, the ones I consider most important and would most recommend. There’s a good list on stack overflow too, if you prefer the wisdom of crowds to the wisdom of me.

Code Complete, Steve McConnell

This is the book that took me from enthusiastic amateur to professional. It covers the programming-in-the-small that you will do every day for the rest of your career: Naming variables, writing for loops, that type of thing. I know, you know how to write a for loop already.

This book will make you better at the small things.

Code Complete: A Practical Handbook of Software Construction

The Art of Unix Programming, Eric S. Raymond

It took me a very long time to read this book. I would pick it, get a few pages in, have an epiphany, and go re-write some things.

Unix is the only constant in our world. The programming language you use will change many times, the tools you use will change all the time, and even SQL is not as much of a constant as it once was. But Unix will always be there for you. Improving your Unix knowledge is the single best investment you can make as a programmer.

But this is not just a book about Unix. It’s a book about the philosophy of Unix, about The Way, and it intends to bring you enlightenment in the Zen Buddhism sense.

For me at least, it did.

The Art of UNIX Programming

The Linux Programming Interface, Michael Kerrisk

This is the Linux grimoire, the spell book with all the spells. It’s over $60, 1500 pages, and you must never get it wet or read it after midnight.

Pretty much everything interesting you do in Linux (open a file, write to a socket, start a process, sleep. allocate memory, everything) is a syscall. This books is all the syscalls, and extensive information around them.

It will answer all your questions.

The Linux Programming Interface: A Linux and UNIX System Programming Handbook

December 12, 2013

Go: How slices grow

Posted in Software at 05:49 by graham

In Go (golang) what happens to memory when you append to a slice?

If there’s enough space in the slice’s backing array, the element just gets added. If there’s not enough space, a new array is allocated, all the items are copied over, and the new item is added at the end. The interesting part is allocating that new array. And here’s the answer:

Go slices grow by doubling until size 1024, after which they grow by 25% each time

This is an implementation detail and may change. The above is correct for Go 1.1 and 1.2.

Try it out:

package main

import "fmt"

func main() {
    var x []int  // Same as x := make([]int, 0)
    for i := 0; i < 100; i++ {
        fmt.Printf("%d: %p cap %d\n", i, x, cap(x))
        x = append(x, i)
    }
}

Read the rest of this entry »

December 8, 2013

My setup: Hardware

Posted in Behaviour at 00:46 by graham

Here’s my current setup, and the hardware I got, in case you were curious. And because I know I’ll be curious in a few years.

  • Aeron chair: Are the any other type of office chairs? I don’t think so. At least there shouldn’t be. You can usually get one cheap from a failing startup in your area.

  • INGO Ikea sitting desk: A plenty big enough desk, of solid wood, and cheap. It’s marketed as a kitchen table. I’ve owned three of these so far.

  • BJORKUDDEN Ikea standing desk: A little bit too short for me, so I have some phone books on top. Getting a standing desk the right height is tricky, because your legs don’t adjust, unlike a chair, so it has to be perfect. This as close as I could find without spending a fortune. I stand about 50% of the time when I’m programming, but not 50% of the day. Some days I mostly stand, others I mostly sit. Standing after lunch helps a lot in overcoming the post-lunch coma. The Bjorkudden is marketed as a bar table.

Hardware

Read the rest of this entry »

« Previous entries Next Page » Next Page »