February 18, 2006

Types of question

Posted in Behaviour, Society at 15:59 by graham

From a Greenpeace workshop

  • Open questions invite the other person to express their thinking freely rather than allowing a “yes” or “no” response. Open questions usually begin with Who, Why, When, Where or How. For example:

    • “How much petrol do you use per week ?”
    • “Where do you normally get your petrol from ?”
  • Closed questions can be answered with a yes or no. They tend to close a conversation down. However, they can be useful for checking facts, clarifying a point, or deliberately finishing a conversation. They can also control the length and form of the reply and allow a “yes” or “no” or brief response. They can be useful on the streets, especially as an opening line.

  • Multiple Questions combine many questions together, making it unclear which question to respond to. People can become confused. Less is more ! For example:

    • “Have you seen, heard or know about Greenpeace ?”
    • “Do you think Esso is bad, or just like the others, or what ?”
  • Leading Questions reflect an assumption, or give an obvious direction to the reply, indicating the desired response. Leading questions hint at what you want the answer to be. They don’t allow a person freedom of expression, or require them to really engage or think. They should usually be avoided and certainly abandonded if they are met with any resistance. For example:

    • “You like peace don’t you ?”
    • “It’s the only way, isn’t it ?”
  • Hypothetical Questions can be good for opening up options and helping people to explore possibilities. However answers bear little relation to what would actually hapen. For example:

    • “If I could prove to you that Esso have absolutely no commitment to the environment, would we get your support ?”
  • Probing Questions are good for following up information already received but can lead to a lengthy conversation which should be avoided! For example:

    • “Why do you say that ?”

    More details on the campaign to boycott Esso

Cal Henderson – Building Flickr

Posted in Future of Web Apps, Software at 14:39 by graham

These are the notes I took during Cal Henderson’s talk on Building Flickr at the Future of Web Applications conference in London on Wednesday 8th February.

Lars Ploughman posted a great mind map of the talk.


  • Try and figure out what people need; this is not always what they say they need.
  • Network effect: When people get their friends to sign up, it makes their experience better.


  • Instead of always grouping data by user, we can slice it in other interesting ways:
    • Most recently uploaded
    • Most popular
    • By tag
    • By weighted ‘interestingness’ algorithm

Open APIs

By API the Web 2.0 people mean a web service API: REST, SOAP, XML RPC, etc. You need an API to for your web client anyway, so clean it up and make it public.

The evolution is: web site -> web application -> web service

Clean URLs

Don’t expose internals or physical structure on the URL. They should reflect a consistent, logical structure, the way the user thinks about it. mod_rewrite enables this. URLs should never change, otherwise you break links.


It’s Asynchronous. It streamlines user interactions.


Internationalisation and Localisation. Store, present and receive all textual data as UTF-8.

Desktop / Platform integration

Backed by the API. With an API for your app, you can write desktop apps. Some interactions on the web are difficult and/or slow (such as uploading lots of pictures). Desktop apps can have drag and drop for example.

It doesn’t have to be a full fledged desktop app – use Bookmarklets, Firefox plugins, etc.

Integrate with e-mail; every user has it and uses it daily. It’s often difficult to get data off mobile phones, but they can do e-mail, so provide an e-mail uploader. Send e-mail notifications.

Mobile phones

They mostly support XHTML-Mobile, a cut down version of XHTML. Build custom pages for mobiles because the screens are so small. Re-think what is on the page. Cut down as much as possible.

Open data

Allow users to import / export their data. Include the meta-data as well. This makes people feel safer about using your service.

Open content

When a user uploads something, they still own it. They still control the licensing.


Joshua Schacter – del.icio.us – Things we've learned

Posted in Future of Web Apps, Software at 14:05 by graham

These are the notes I took at Joshua Schachter’s talk at the Future Of Web Applications (futureofwebapps) conference, in London on Wednesday 8th February 2006.

Lars Ploughman posted a great mind map of the talk.

  • del.icio.us started as a text file of 25 000+ links on Joshua’s computer. Each one had a descriptive sentence, that got shortened down to a single word -> a tag. When building an application, it must solve a problem you have. You need passion.

  • Cross browser compatibility is difficult

  • Monitor your app.
  • Optimize: Add database indexes. Cache. Monitor user behaviour to find out what they do. This allows you to optimize certain parts – for example no-one ever goes to search results page 100. Figure out where latency is acceptable and use that; for example the RSS feed can be a few minutes behind the live site.
  • Aggregation of attention is interesting: What are people looking at, etc. Top 10.
  • Your application will be used in unexpected ways, so react to it. Monitoring helps.
  • Learn the ins and outs of Apache, including mod_rewrite.
  • Put a proxy in front of Apache – for example pound or perbal.

  • Build an API, and make sure it is easy to use. Avoid SOAP, CORBA, etc. Stick to simple XML. The API should allow uses to save their data offline, to script your application, etc.

  • Don’t expose internal ids, especially if they are consecutive. Otherwise people will try and iterate over them to get all your data, and overload your site.
  • Don’t add features that already exist elsewhere. For example your site doesn’t need a messaging system – use e-mail instead.
  • Add features that people will use; this is not always what they ask for. Get to the root of each feature request – what exactly is the user trying to do, and is there a better way of doing that than what they asked for.
  • RSS is the web’s native style for a list of links. Put RSS everywhere you have a list. It’s another way for users to get data from your system. del.icio.us has more RSS than HTML traffic.
  • Sensible URLs: They get passed around, bookmarked, mailed, etc. They should be user-friendly, not developer friendly. Don’t include session ids, or implementation specific details on the URL.
  • Release early and often. No closed beta.

  • When you block spammers, don’t let them know. Don’t display an error message, or change visible behaviour. Otherwise they start trying different methods.

  • Understand user motivation. The application has to be useful for user number 1. Users are selfish. Ideally people use the system and evangelise it. They come for themselves not for the data others created.
  • Measure how people react to new features and how long they keep using a feature. Rely on what people do not on what they claim.
  • Do user acceptance testing. Everyone involved in the project should watch this. If you can’t afford a test lab (with a two-way mirror for the team to stand behind), do ghetto testing: Buy people coffee in the local Internet cafe and show them the application. Don’t give them goals, just let them be normal users.
  • Speak the users language: Only Netscape / Firefox users call them bookmarks – IE users call them favorites.
  • Don’t force users to register. Or if you really need to, delay it as long as possible. Give as many public features as possible. Make registration short and afterwards send them back to where they were.
  • Understand the Internet design grammar and choose when to follow it and when not to. For example clicking the logo in top left corner goes back to the home page.
  • Remember that the data is not yours, it belongs to the user, and you’re just borrowing it. Allow users to export their data, delete themselves completely from the system, etc. Having an API helps with this.
  • The clients to your web app aren’t just browsers. Think of everything that can take data from your server: RSS readers, iCal clients, Excel, widgets (Konfabulator and Dashboard) etc.
  • Build a tool, not a community. Unless you are explicitly trying to build a community.


February 16, 2006

Why you need a cluster

Posted in Software at 14:37 by graham

If your live web application has more than a couple of users, and they don’t both sit right next to you, you need to cluster it. This is not for performance reasons – a single server can easily cope with the load on most web applications. It is not even for reliability reasons – problems are most likely to be in the application code which runs on all cluster members, than in the infrastructure. You need to cluster for upgrade and maintenance reasons.

Cluster for manageability

A dispatcher should sit on your public web address, and proxy requests to one of many (most likely two) back end servers. When you need to release a new version of your application, upgrade your server, or even your hardware, you can take that server out of the cluster. Once the server is upgraded, it goes back in the cluster and the other one comes out.

Sticky sessions instead of session clustering

If your application does not use sessions, you can let the proxy choose a different back end server for each request. If you have sessions, you could replicate the session. This means the back end servers communicate to share session data. This can be done via the database or via sockets or shared memory. This is complex to setup and slows your application. Much easier and faster is sticky sessions. A good proxy allows you to pin each session to a server, so that one users stays on one server for the duration of their session.

Front end dispatcher software

Other benefits

  • You won’t have to get out of bed if one of the servers goes down

  • Get the proxy to take care of https, meaning your back end servers don’t need to.

  • You can do last minute testing by taking a server out of the cluster, releasing the new version of your application to it, and browsing straight there to test. It’s a live, private server !

  • You can tell the security officer that your application server is not directly accessible from the outside world.

  • For extra credit, you can setup a backup proxy to do IP address takeover if your proxy goes down, using Heartbeat. Example here

    Happy clustering !

February 15, 2006

Goodbye mod_jk hello mod_proxy

Posted in Software at 14:18 by graham

I’ve been using Apache as a front end to a cluster of Tomcat servers for a while. I’ve always used mod_jk2 or mod_jk; I even wrote an article about it. Not any more.

The most recent Apache comes with mod_proxy, mod_proxy_balancer and mod_proxy_ajp, which do everything I used mod_jk to do.

Setup mod_proxy_balancer

First you need to make sure they are included when you compile Apache:

./configure –prefix=/usr/local/apache2.2 –enable-proxy –enable-proxy-ajp –enable-proxy-balancer –enable-proxy-http

Then in your httpd.conf you need: BalancerMember ajp://bill.darkcoding.net:8101 route=server1 BalancerMember ajp://ben.darkcoding.net:8102 route=server2

    <location /myApp>
        ProxyPass balancer://myCluster/myApp stickysession=JSESSIONID

    <location /balancer-manager>
            SetHandler balancer-manager

        AuthType Basic
        AuthName "Cluster manager "
        AuthUserFile /usr/local/apache2.2/conf/.htpasswd
        Require valid-user


The above defines a cluster and adds your two servers to it. Then it proxies your app location using that cluster. Finally it defines a URL for the balancer manager. This is the replacement for jkmanager. The stickysession value needs to be the name of the session url param name or cookie. The above is correct for Tomcat. Note that the session id is case sensitive.

Finally in you Tomcat server.xml you need to include a jvmRoute to match the route defined above. This is exactly the same as when using mod_jk, so you probably already have it. It should look somewhat like this:

   <engine name="Catalina" defaultHost="localhost" debug="0" jvmRoute="server1">

and similarly for server2.

Eh voila !

If it’s not working change your LogLevel in httpd.conf to debug and restart Apache. If you get this error:

Found value (null) for stickysession JSESSIONID

then Apache is not finding your route on the session id. Check the spelling and capitalisation of the stickysession entry. Using the Firefox developer toolbar to check you Cookie. It should look something like JSESSIONID=4f39cBLAH.server1. The route is appended to the session id after a period.

There is also apparently a problem with disabled servers coming back to life. See this thread

Happy proxying !

February 1, 2006

Javascript and PHP credit card generator

Posted in Credit card, Software at 22:39 by graham

I have ported the credit card number generator to PHP and Javascript. The files can be found in the same ZIP as the original Python program.

Go here or follow the link under Pages on the right.