May 24, 2014

Sync, a Unix way

Posted in Software at 05:49 by graham

Ever since Dropbox, I’ve been searching for a self-hosted, secure (and now Condi-free) way of keeping my machines synchronised and backed up. There are lots. I tried many, wrote a couple myself, but none were exactly what I wanted.

My problem was thinking Windows, looking for a single program. Once I started thinking Unix, looking for modular components, the answers were obvious.

Storage

First we need a remote master storage to sync against, somewhere to backup our files. And we want that exposed as a local filesystem. I use the most obvious answer, sshfs:

sudo apt-get install sshfs
mkdir -p /home/graham/.backup/crypt  # Why 'crypt'? Read on.

sshfs server.example.com:backup /home/graham/.backup/crypt

You can use any storage that can appear as a filesytem, such as FTP (via curlftpfs), NTFS, and many others.

Encryption

There’s two kinds of data: public data, and encrypted data. We want the second kind. Just layer encfs:

Read the rest of this entry »

May 4, 2014

GopherCon 2014 favorite talks, notes

Posted in Software at 19:39 by graham

My favorite talks at GopherCon 2014:

  • Peter Bourgon: Best Practices for Production Environments Soundcloud were an early Go adopter, and this talk is their distilled learnings from two years of Go: Repo structure, config, logging, testing, deployment, and lots more. The one talk you need if you’re starting (or running) a significant Go project, and you want to do it right.

  • Petar Maymounkov: The Go Circuit: Towards Elastic Computation with No Failures Stick with this one. It starts off quite academic, but gets fascinating very fast. He models whole companies as a distributed system (based on CSP), then builds a language-agnostic cluster programming library where the API is a filesystem The Circuit. One of the highlights of the conference for me was building a filesystem with Petar in the hallway.

  • John Graham-Cumming: A Channel Compendium John is the author of those great in-depth Cloudflare blog posts. Solid talk about Go channels. nil channels always block, so you can ‘disable’ a select clause by setting a channel to nil. Closed channels never block. Heartbeat is just time.Tick, timeout is time.After. Go programs are small sequential pieces joined by channels.

Those are the three talks I enjoyed most. Here are my general notes on the conference and a few of the other talks. Read the rest of this entry »