January 31, 2015

Raw sockets in Go: Link layer

Posted in Software at 21:55 by graham

Continuing our dive into the Internet Protocol Suite from Go (See part 1 Raw sockets in Go: IP layer), we are going to the link layer, so we can see the IP headers. This will also allow us to craft our own IP headers, or handle address families outside IP. We’ll send ping packets (ICMP echo request) and watch the kernel’s response.

Receive

This isn’t wrapped in Go, so we need a syscall. Otherwise it’s very similar to the IP layer in part 1, and pretty similar to the C equivalent.

On the first line of main we request the AF_INET family, meaning IPv4. We could ask for a different address family (AF_* constants) – here’s a list of address families. Most of the protocols in that list are rare (AF_IPX, AF_APPLETALK, etc). We’re in a IP world today.

Other useful address families:

  • AF_INET6 for IPv6.
  • AF_UNIX for unix domain sockets. It is used in net.DialUnix and net.ListenUnix. The POSIX name for AF_UNIX is AF_LOCAL, but Go largely sticks to AF_UNIX. They are equivalent.
  • An odd / interesting one is AF_NETLINK, which is for talking to the kernel. Read about it man 7 netlink or at Linux Journal. Docker has a netlink package.

The second parameter, SOCK_RAW is what makes this a raw socket, where we receive IP packets. SOCK_STREAM would give us TCP, SOCK_DGRAM would give UDP.

The third parameter filters packets so we only receive ICMP. You need a protocol here. As man 7 raw says “Receiving of all IP protocols via IPPROTO_RAW is not possible using raw sockets”. We’ll do that in the next post in this series, at the physical / device driver layer.

Build and run it as root (only root or CAP_NET_RAW can open raw sockets). In a different window ping localhost. You should see something like this:

45 00 00 3C EA FF 40 00 40 06 51 BA 7F 00 00 01 7F 00 00 01 …

This is the IP Header. First byte 45 is 4 for the IP version (IPv4), and 5 for length of this header (5 32-bit words), and so on. This is just like the receive example in the previous post except that we also see the IP header.

Try replacing IPPROTO_ICMP in the Socket call with IPPROTO_TCP, and wget localhost. The first 20 bytes will be similar (the IP header), then you should see a TCP packet, and finally HTTP.

Read the rest of this entry »

January 25, 2015

Continuous Delivery: my notes

Posted in Software at 05:44 by graham

Continuous Delivery, by Jez Humble and David Farley is about three big ideas to get your code into production more reliably:

  • Make a deployment pipeline: commit -> unit test -> acceptance test -> … -> deploy -> release
  • Automate everything.
  • DevOps. Project team should be mix of development, operations and quality assurance / test. Involve operations (sysadmins) from the start.

Stages of the deployment pipeline:

Stage 1: Commit Tests

Trigger off a version control push. Usually happens in Continuous Integration server.

  1. Static analysis (lint, code metrics like cyclomatic complexity & coupling)

  2. Compile

  3. Unit test (output code coverage):

    • Check that a single part of the app does what the programmer intended.
    • Should be very fast.
    • Do not touch the database, filesystem, frameworks, system time or external systems. Mock or stub these, or use in-memory db.
    • Avoid the UI.
    • Try to avoid testing async code. Should never need to sleep in unit tests.
    • Include one or two end-to-end tests to prove app basically runs
  4. Package a release candidate. Bake in version number.

    Use OS’s packaging tools (deb, rpm). Operations team will be familiar with it, all the tools support it.

  5. Push release candidate to artifact store (a file system, or full fledged artifact repository)

    Read the rest of this entry »