February 25, 2007
Posted in Society, Software at 14:05 by Graham King
Last weekend I attended BarCampLondon2. Bar Camp is an unconference, where a group of like minded people get together, everyone presents a topic / session, and we all hang out and discuss things. On the morning of the first day you write your topic on a card and stick it on a board where times and rooms are layed out in a grid.
The most interesting sessions for me were the following:
Rhys Jones on Offline web applications
Rhys gets the train from Wales to London regularly, and needed to use webapps in general, and Google Calendar in particular, whilst on the train without a net connection. He evaluated various ways of doing this and ended up writing a proxy that, when offline, pretends to the Javascript in Google Calendar that it is the server, and when online replays all the actions it intercepted. He’s calling it Web2OS and it’s really worth a look.
Offline web apps is something we’re going to have to solve. Firefox 3 will provide native support for this. Ultimately web application builders will need to figure this out. I expect a lot more interest in this area in the future.
Rhys’ proxy also allow ‘local mashups’. For example your local proxy can read what music you are playing on your machine (assuming your player has an API), and then provide a local URL where it display Flickr images for the song title. Or it can read your address book, and display a Google Map with your contacts on it. Really cool stuff.
Jay Caines-Gooby on building an Amazon EC2 AMI to run webapps, entitled Making Rails Elastic
Jay talked about how instead of renting a new dedicated server, his company is going to use Amazon’s Elastic Compute Cloud to rent server power as they need it. Building the AMI (Amazon Machine Image) is easy enough on Linux - you setup a directory, chroot to it, install and get everything working as you want it, then come out of the chroot and package the image up.
The real challenge is that whenenver your machine is restarted, all disk storage is wiped. The only place you can store something permanently is in Amazon’s S3. Jay talked about ways of mounting S3 as a filesystem, using S3 InfiniDisk, S3FS or S3FS-Fuse; of these InfiniDisk sounds the most mature.
Running my server off EC2 is something I would love to do, but I think it’s a little too early adopter right now. Definitely one for the future. If this takes off hosting companies will have to following the EC2 model or retire.
Simon Willison with Open ID Explained
OpenID is going to be the main authentication system on the Internet, so I’m really pleased I finally get it ! Next time you’re building a webapp and thinking of writing the authentication section for the millionth time, including the ‘I forgot my password’ page and the dready rest of it, stop. Just add OpenID and be done with it. Your users will thank you.
Tom Coates on social web apps Greater than the sum of its parts
Tom is a visionary, and a great speaker. He really got to the heart of how social web apps (like flickr, facebook, orkut, linkedin, wikipedia, last.fm, etc) work. As I don’t have any plans to build one of these (it takes a tremendous amount of self belief and a skillful marketing operation to create an online community from thin air), I watched his talk more as a lesson on how to present.
Janette Girod on Optimising the everyday - Finding Flow
How to get into the zone and work more productively. The best tip for me was splitting up your work time into 48 minutes total concentration / no distractions / email off / IM off / go for it, and then 12 minutes break, where you get up, walk around, talk to people, and catch up on your communications and web browsing.
Gareth Rodgers on Integrating Mobile Messaging with your web applications.
The crux of this was that the kannel project can drive your mobile phone to send and receive SMS messages, and with mbuni it can do MMS (pictures, audio clips, videos, etc). You connect a mobile phone (via USB or Bluetooth) to your server, and kannel provides an API for you to send and receive messages.
Most server setups will have an automatic way of sending a message to the administrator if something goes wrong. Nearly all of these will use a third party gateway, charging per SMS. Kannel provides a way to do this without a third party provider. The ability to receive messages also opens up the way for more mobile interaction with your application. Now if I could only convince my hosting provider to plug my phone into the server !
A few days after that talk Tim Stevens pointed me to the British Telecom SDK, which looks like it provides an easy way to send SMS messages.
Papervision 3D a Flash 3D engine.
Wow ! Proper 3D in Flash. This should make Flash games a lot more fun !
Improv workshop by James Aylett
Not about technology, but one of the best sessions of the weekend. James got the group to try out some Improvisational Theatre. I found it challenging and very funny.
I presented A Jabber Wonderland which I have written about previously. Other presentations can be found on the BarCamp Wiki or on Slideshare barcamplondon2 tag.
All in all it was a fantastic weekend. Many many thanks to the fantastic barcamplondon2 organisers.
February 16, 2007
Posted in Software at 00:11 by Graham King
Webapps are often built and managed by single person or small teams. For example, I run 11 sites and web apps, just in my spare time, on one virtual server. I need to know if that server is up. We need a lightweight, low/zero maintenance way to monitor applications and servers. Mainly we are concerned about presence of our application. Is it there ?
Where else does presence matter ? Why, in Instant Messaging applications of course. Currently we record presence of our friends in our messenger clients. Well, servers can be our friends too !
Jabber / XMPP
XMPP is the eXtensible Messaging and Presence Protocol. Originally know as Jabber (before it was standardised), it is an open, XML-based protocol for near-real-time, extensible instant messaging and presence information. It is currently used on thousands of Jabber servers across the Internet and is used by millions of people worldwide. The Internet Engineering Task Force (IETF) formed an XMPP Working Group in 2002 to make Jabber / XMPP an official standard.
When Google built Google Talk, they chose XMPP as the core protocol.
There are free (speech & beer) servers, server software and libraries for most languages. Sounds perfect.
Brief Jabber Glossary
The Jabber HOST is the domain name of your Jabber server. For example ‘jabber.org’, ‘gmail.com’ (if using Google Talk), etc.
A JID (Jabber ID) (from Wikipedia):
Every user on the network has a unique Jabber ID (usually abbreviated as JID). To avoid the need for a central server with a list of IDs, the ID is structured like an e-mail address with a username and a DNS address for the server where that user resides separated by an at sign (@). For example, if the Montague family ran a Jabber server on the domain montague.net, then the user romeo might be assigned the address romeo@montague.net.
Since a user may wish to login from multiple locations, the server allows the client to specify a further string known as a resource, which identifies which of the user’s clients it is (for example home, work and mobile). This may then be included in the JID by adding a forward slash followed by the name of the resource. For example the full JID of Romeo’s mobile would be romeo@montague.net/mobile. Messages that are simply sent to romeo@montague.net will go to all clients that Romeo has logged in at the time, but those sent to romeo@montague.net/mobile will only go to his mobile.
The RESOURCE is the last part of the JID, usually ‘Home’.
A simple monitor in Python
Python’s main XMPP library is xmppy.sourceforge.net: Send a message.
conn = xmpp.Client(HOST)
conn.connect()
conn.auth(USERNAME, PASSWORD, RESOURCE)
conn.sendInitPresence()
conn.sendMessage( xmpp.Message(' user@jabber.org', 'Hello User') )
Replace ‘user@jabber.org’ with your jabber id, and fill in HOST, USERNAME, PASSWORD and RESOURCE.
USERNAME is the JID of the jabber account you created for your machine, for example ‘mymachinexyz@jabber.org’. RESOURCE is usually ‘Home’.
If you are using Google, replace the ‘connect’ line with:
conn.connect( server = ('talk.google.com',5223) )
If you just want to run the monitor, JIMBo (see later) is linked at the end of this article.
The connection will automatically use SSL / TLS if it is available. The Google Talk, for example, will use it.
Next you need to allow yourself to view the machine’s presence:
roster = conn.getRoster()
roster.Authorize(' user@jabber.org')
Add ‘USERNAME’ as a Buddy in your Jabber IM client and you’ll know when your machine is online or offline.
Do the above as a thread in a Java server, and you know when the server is running. Set an alert when your buddy ‘USERNAME’ goes offline and you have server monitoring !
You don’t even need to run your own server: There are lots of public Jabber servers (such as jabber.org), Google Talk has some, and Google Apps For Your Domain will even run one for you on your domain !
A simple bot
But wait, there’s more. Now that our machine is our friend, we probably want to chat to it:
def onMessageReceived(conn, mess):
sender = mess.getFrom()
text = mess.getBody()
print str(sender) +" send: "+ str(text)
conn.RegisterHandler('message', onMessageReceived)
We can do anything we want on our server, remotely. We have a bot.
An aside: Jabber for you
Jabber servers have gateways to other IM’s: AIM / Yahoo / MSN / ICQ / etc. Get a Jabber id and you don’t need a multi-protocol client anymore, the gateways do it for you.
With your own server, or Google Apps For Your Domain, your Jabber id can be the same as your e-mail address !
Machines Jabber the world
If we can talk to our server, so could another server. And when two machines talk to each other, enterprise types call the bit in the middle a Messaging Solution ! Consultants will want you to buy an Enterprise Service Bus, to form the basis of your new Service Oriented Architecture. Ignore all that.
Jabber has Publish / Subscribe.
Jabber / XMPP makes it easy to wire machines together. The servers and protocols have millions of users. Find an ESB that can boast that.
And you can have redundancy too: ejabberd has it built in, or get your machine to add all the redundant servers into a Buddy Group (named after the service they provide for example), then cycle through all of them until it finds one that is online !
How deep is this rabbit hole ?
The more you look into Jabber, the more interesting it gest. Probably the most widely deployed Jabber server, ejabberd, the one running at jabber.org, is written in Erlang.
Erlang, developed then open-sourced in 1998 by Ericsson Computer Science Laboratory, is a soft realtime, declarative, functional language for concurrent, distributed systems. In other words, it’s not like most languages you will have used (most likely procedural and object-oriented languages), except at college if you studied Computing.
But apparently Erlang has amazing light-weight thread support.
And there’s a web server written in Erlang: YAWS (Yet Another Web Server). And when you lookup YAWS, you find this graph:
Source: http://www.sics.se/~joe/apachevsyaws.html
On the left is web server throughput in KBytes/second. Along the bottom is number of concurrent requests on a web server. In green is Apache and in red is YAWS. What this is saying is that (on the hardware of the test) Apache dies where there are over 4,000 concurrent requests. YAWS is still going at 85,000. All the detail here: Apache vs. Yaws.
So that’s pretty interesting too.
Download JIMBo
Download the Jabber Instant Monitoring BOt / JIMBo. Linux / Unix only. Installation instructions are in the zip. GPL license. v0.2.2 now includes auto-reconnect, proper daemon behaviour, and Gentoo and RedHat startup scripts
I will be presenting this at BarCamp2London this weekend
Feed
RSS / Atom feed