From their homepages:

Node.js: Evented I/O for V8 JavaScript. (A framework for building completely non-blocking servers in Javascript)
NPM: A package manager for node.
CoffeeScript: A little language that compiles into JavaScript
Express: High performance, high class web development for Node.js
Jade: Node Template Engine
Redis: An open source, advanced key-value store

In this guide I will take very small steps so that you can verify that you are check whether you are still on track.
The result is an extremely performant, scalable and lightweight alternative for web development.

So I installed a brand new Ubuntu 11.4 virtual machine and got started. After following some tutorials, googling and some tweaking this is the step-by-step guide I came up with.

Installing Node.js

$ sudo apt-get install python-software-properties
$ sudo add-apt-repository ppa:jerome-etienne/neoip
$ sudo apt-get update
$ sudo apt-get install nodejs


Installing npm

$ apt-get install curl
$ curl http://npmjs.org/install.sh | sudo sh


Node.js Hello World

$ mkdir helloworld
$ cd helloworld
$ vi helloworld.js


insert

var http = require('http');
http.createServer(function (req, res) {
  res.writeHead(200, {'Content-Type': 'text/plain'});
  res.end('Hello World\n');
}).listen(1337);
console.log('Server running at http://127.0.0.1:1337/');

$ node helloworld.js

check http://127.0.0.1:1337

Add Coffeescript

Writing all that javascript is going to lead to lots and lots of brackets, semi-colons and parantheses. Let’s use Coffeescript to make our code readable again.

$ sudo npm install -g coffee-script
$ cd ..
$ mkdir hellocoffee
$ cd hellocoffee
$ vi hellocoffee.coffee


insert

http = require "http"
http.createServer( (req, res) ->
  res.writeHead 200, {"Content-Type": "text/plain"}
  res.end "Hello Coffee!!"
).listen 1337

console.log 'Server running at http://127.0.0.1:1337/'

$ coffee -c hellocoffee.coffee
$ node hellocoffee.js

And check http://127.0.0.1:1337 again.

Adding Express

Node.js is a very powerful, but it still requires quite a bit of boilerplating to use it for web development. That’s where Express comes in. It is build on top of another middleware layer called Connect and gives you much better support in handling requests, rendering, templates and responses.

$ cd ..
$ sudo npm install -g mime qs connect express
$ mkdir helloexpress
$ cd helloexpress
$ express
$ npm install -d
$ node app.js


check http://127.0.0.1:3000

vi app2.coffee

insert:

express = require('express')app = express.createServer()

# Setup configuration
app.use express.static(__dirname + '/public')
app.set 'view engine', 'jade'

# App Routes
app.get '/', (request, response) ->
  response.render 'index', { title: 'Express with Coffee' }

# Listen
app.listen 3000
console.log "Express server listening on port %d", app.address().port

$ coffee -c app2.coffee
$ node app2.js


And check http://127.0.0.1:3000 again.
Now that we have Coffeescript working with Express let’s add sessions.

Add session support

$ vi app2.coffee

insert after app.use express.static(__dirname + ‘/public’)

app.use express.cookieParser()
app.use express.session {secret: "Coffeebreak" }

insert after app.get ‘/’, (request, response) ->

request.session.views++

replace

response.render 'index', { title: request.session.views + ': Express with Coffee and sessions' }

$ coffee -c app2.coffee
$ node app2.js

Check http://127.0.0.1:3000 and refresh a few times. You should see the counter increase.

Install Redis

Open a new terminal.

$ wget http://redis.googlecode.com/files/redis-2.2.11.tar.gz
$ tar -zxvf redis-2.2.11.tar.gz
$ cd redis-2.2.11/
$ sudo apt-get install build-essential
$ make
$ ./src/redis-server


Add redis session to node

$ npm install -d redis connect-redis
vi app2.coffee

insert after express = require ‘express’

RedisStore = require('connect-redis')(express)

change

app.use express.session {secret: "Coffeebreak", store: new RedisStore, cookie: { maxAge: 60000 } }

$ coffee -c app2.coffee
$ node app2.js


In the Redis terminal you should now see something like:
[25489] 24 Jun 04:57:11 - DB 0: 1 keys (1 volatile) in 4 slots HT.
[25489] 24 Jun 04:57:11 - 1 clients connected (0 slaves), 799032 bytes in use


Check http://127.0.0.1:3000 and refresh a bunch of times
kill and restart node

$ node app2.js

Check http://127.0.0.1:3000 again and refresh a few more times
Note that you started where you left off.

And that brings us to the end of this guide. As you can see the stack is very powerful and extremely small, lightweight and clean. With all the session information in the Redis store the availability of the entire systems comes down to the availability of Redis. And with these guys working on a proper clustering solution it will be very interesting to see where all this is going in the near future.

Introduction

I want to skip the introduction because a lot has been written about websockets and you probably want to get started as soon as possible. Check out the sources section for more information. For now you only have to know that it’s fast, it has almost no overhead, keeps a permanent connection open (so the server can push data to the client, no polling required!) and best of all: it’s very easy.

Installation

When you’re using maven 2+ you don’t have to download anything. Just start a maven war project and add the jetty-maven-plugin (we used version 7.2.1.v20101111), add a dependency to jetty-websocket (inside the project element) and you’re ready to get started.
<dependency>
<groupId>org.eclipse.jetty</groupId>
<artifactId>jetty-websocket</artifactId>
<version>7.2.1.v20101111</version>
</dependency>

Dissecting the WebSocketChatServlet example

A sample application is included with Jetty. On this blog this sample is explained and in the end we used it to see what was happening and of course see some communication between client and server. After that we stripped everything and started over again to see what was actually happening.

Creating your own servlet

Since I used Struts and Spring MVC in the past and always copied the servlet configuration it was a while ago I added my own servlet to web.xml, so here is a litte snippet to help you (this probably was one of the most difficult parts ):
<servlet>
<servlet-name>hw</servlet-name>
<servlet-class>com.xebia.ws.HelloWorldServlet</servlet-class>
<load-on-startup>1</load-on-startup>
</servlet>

<servlet-mapping>
<servlet-name>hw</servlet-name>
<url-pattern>hw.do</url-pattern>
</servlet-mapping>
The HelloWorldServlet extends WebSocketServlet and you have to implement the doWebSocketConnect method. This method returns a WebSocket. For now it’s enough to do a new HelloWorldWebSocket.

HelloWorldWebSocket implements WebSocket. The onConnect method receives an Outbound object. With this object it is possible to send data back to the client. The Outbound object is a field on the WebSocket object so we can use it in the onMessage method.

The onMessage method:

public void onMessage(byte opcode, String data) {
    outbound.sendMessage("pong");
}

When we send data in javascript (with send()) the server will respond with “pong”.

The Javascript side

The Javascript is also very simple. I added the jQuery plugin to add data to a div with id bla, but your free to do it the old-fashioned way.

var ws=new WebSocket("ws://10.0.0.230:8080/hw.do");
ws.onmessage = onMessage;
ws.onopen = onOpen;

function onMessage(m) {
    $("#bla").append(m.data + "<br/>");
}

function onOpen() {
    console.log("connection open")
}

function send(){
console.log("sending ping");
    ws.send("ping");
}

For testing we’re using Safari, it has a really nice console (available under the menu Develop, Show Error Console). By typing send() in the console we’re sending a ping which will respond with a pong. The response is logged to the div with id bla.

It’s even possible to write a simple websocket client with nothing more than the console:

var m=new WebSocket("ws://10.0.0.230:8080")
m.onmessage=function(m){ console.log(m.data); };

This piece of code prints the payload of any received message. Sending is done with m.send(“data to send”);

Problems

Thinking http is the protocol of websockets

We used the browser to test whether the WebSocketServlet was working. Of course websockets uses the ws:// or ws:// prefix and we used http://. I made this mistake a couple of times.

WebSocket is not ready

We tried to send data over the socket, but got an INVALID_STATE_ERR. It appeared the WebSocket isn’t ready immediately after creating. After the WebSocket is ready the onopen function is called and then it’s ready to send and receive. The readyState field on a WebSocket is a useful field to check the status of the socket (0=connecting, 1=open, 2=closing, 3=closed). So don’t try to do anything with the websocket before onopen is called.

WebSocket disabled in Firefox 4

While we were coding away Dan North paid us a visit. In a very cool improvisation style presentation he pointed us to an article about some security issues with websockets.

Conclusion

Websockets is a very cool technology and I’m very happy it was a subject of the techrally. Feel free to ask any questions. At least 5 people at Xebia are now websockets experts and even more listened to two presentations about it.

Sources

http://blogs.webtide.com/gregw/entry/jetty_websocket_server
http://dev.w3.org/html5/websockets/
http://blog.dannorth.net/
http://www.slideshare.net/spagalloco/introduction-to-websockets
http://docs.codehaus.org/display/JETTY/Maven+Jetty+Plugin
http://hacks.mozilla.org/2010/12/websockets-disabled-in-firefox-4/
HTML5 Series: Part 6: WebSockets

Embedding Wave

You can embed a Wave into a web page by adding some simple JavaScript code.
The Wave Embed API provides the WavePanel object which can hold a wave. You ask the WavePanel to use an HTML element on your web page to show a wave. The conversations on the wave will be visible in the WaveClient.

The steps to embed a wave on a web page are -

1. Load the Embed API JavaScript -

2. Create an HTML element that will contain the embedded wave -

3. Initialize the WavePanel object -

   	function initialize() {
   	       var wavePanel = new WavePanel('http://wave.google.com/a/wavesandbox.com/');
   	      	wavePanel.loadWave('wavesandbox.com!w' + waveID);
         		wavePanel.init(document.getElementById('waveframe'));
       	}
   	...
   	  
   	...
   

The wave is embedded in an iframe created inside the supplied HTML element.
The argument passed to the WavePanel is the Wave server instance. This value is used to set up the URLs used in the iframe created. For the early developer access, it must be http://wave.google.com/a/wavesandbox.com/ (including the trailing slash).

Extending Wave – Robots

Robots are server side programs that can act as participants in a Wave. They can edit content, add users, extract content and post it to an external service thus acting as gateways between a Wave and an external service such as Twitter.
A robot can respond to wave events such as wavelet_blip_created or wavelet_participants_changed. You can also specify a Cron schedule to ask the Wave to contact the robot at regular intervals.

You use a capabilities.xml file to specify what events a robot is interested in. An example from one of the samples is -

There are Python and Java libraries to help you write Robots.

In Java, you can extend AbstractRobotServlet class and implement processEvents(RobotMessageBundle events) to create a wave robot. This class translates the incoming Wave events in the form of HTTP requests into Java method calls, and communicates the changes made back to the wave.

You interact with the wave by using objects provided by the Wave Robot API.
The following lines of code add a new “blip” to a wave -

		Wavelet wavelet = events.getWavelet();
            	Blip blip = wavelet.appendBlip();
            	TextView textView = blip.getDocument();
            	textView.append("Hello World!");

It seems that the only way to deploy a robot at the moment is to deploy it to Google App Engine. Then you can add the robot to a wave using its Wave ID, which is its App Engine application ID followed by @appspot.com. This should change in near future and allow you to deploy the robot to any URL.

See the Robots Tutorial for a step by step guide.

Extending Wave – Gadgets

Gadgets are small programs that add functionality to the Google Wave client. They are written in JavaScript and HTML.
Gadgets use the wave object defined in http://wave-api.appspot.com/public/wave.js file to interact with the wave they are part of. Each gadget has a state object, which is a a map of key-value pairs. You can implement callbacks that are called by the wave whenever the gadget state or the list of participant changes.
The HTML and JavaScript making up the gadget are packaged as an XML file, and can be hosted anywhere on the internet.
See the Wave Gadgets tutorial for example code for a gadget that allows you to run an auction in a wave.

Resources -

Google Wave Robots: Creating a Robot
Google Wave API samples
Wave Gadgets Tutorial
Wave Robot API
Google Wave API forum

At Google I/O 2009, Google unveiled Wave. Wave is a new way of thinking about online conversations.

Consider the following situations -

1. You write a blog post. Somebody comes around and posts a really insightful comment. Now, you or the comment author want to convert that comment into an independent blog post.
2. You have been having a long email discussion with a colleague. Now you would like to invite another colleague to the same. She will need to know the context of the discussion and how it evolved.
3. You email a few colleagues a draft of an article for review. They all email their comments back to you. A lot of them are suggesting the same changes without realizing that they have already been addressed.

With the tools we are using today such as email, blogs, IM etc., all of the above will require some kind of tedious copy – paste, and manual tracking of the changes being made.

Is there a better way?

The Google Wave model tries to provide a better way by doing away with the distinction between email, IM and other forms of online conversations. In this model, an email conversation, a blog post with its comments or an auction with its bids, are just online conversations with multiple participants. At the data model level, all you have is a wave, and you can look at it in many different ways.

Multiple participants can edit a Wave at the same time, and the wave client can show the changes in real time. The changes made to a Wave are stored as a series of operations. You can play the changes back, or revert the Wave to an earlier version. At the data model level, constituents of the Wave form a tree structure. You can take a particular node and spin it into an independent Wave.

Wave providers (servers) give out Wave accounts, and are responsible for sharing Waves with the clients, as well as with other Wave servers. Wave servers communicate with each other using Google Wave Federation Protocol, which is an extension of XMPP protocol. Wave servers use cryptographic signatures and certificates, in addition to transport level encryption provided by XMPP, to authenticate among themselves.

Google Wave Operational Transformation

A key concept at the center of Google Wave is Operational Transformation (OT).

---

Credit: Wikipedia

Many participants can edit a Wave at the same time, and they can see each other’s changes as they are made. These attributes, concurrent modification and low latency updates, are implemented based on OT.
OT is a framework for concurrency control. The document being edited is replicated at all sites. All editing actions are stored as operations that are propagated across all clients, local and remote. Concurrent edits are transformed by the server before being applied and the transformed operations are communicated to all clients. This is very similar to how version control systems like CVS manage merging of simultaneous edits of a document.

From the Google Wave Operational Transformation whitepaper -

Wave OT modifies the basic theory of OT by requiring the client to wait for acknowledgment from the server before sending more operations. When a server acknowledges a client’s operation, it means the server has transformed the client’s operation, applied it to the server’s copy of the Wavelet and broadcasted the transformed operation to all other connected clients. Whilst the client is waiting for the acknowledgment, it caches operations produced locally and sends them in bulk later.

For more technical details on these topics, see Operational transformation, and Google Wave Operational Transformation.

Google Wave Data Model

A Wave is made up of Wavelets. Each wavelet has a unique id within its wave. The Wavelet contains a list of participants and a set of documents. A document in a wavelet has a unique id within its wavelet. It is composed of an XML document, and a set of annotations. Some of these annotations specify the styling of the text content of the document.
Concurrency control and operational transformation are applied at the level of Wavelets. A particular wave user may only have access to a subset of the wavelets in that wave.

The state of a wavelet is defined entirely by an ordered sequence of operations that have been applied to it. Wave clients send these operations to the server to communicate changes to the underlying document, and the server propagates these operations (after a transformation, if required) to other clients and other servers participating in the wave.

Resources –
Google Wave Data Model and Client-Server protocol
Google Wave Federation Architecture
Operational transformation
Wave Protocol forum
Google Wave Developer Blog

In the next blog post on this subject, we will look at different ways of developing with Google Wave.

At the expo Twitter was hot and so were the social networking or the social effects of web 2.0 to drive economy and business. Power of less was the new phrase in vogue which also reflected the mood of participants hearing the news of US jobs losses real time as this four day long event progressed.

Making web work to search local information could be a new mantra to help people who are seeking local information such as price of a watch in nearby stores. As mobile internet becomes more popular, these searches can influence consumers’ behavior when he is looking for a product in the high street. For example, consider Google search where it figures out a possible local context from a search term and displays the local results on the top. Nokia hinted at making location aware mobile devices integrated with smart camera phones that can take advantage of internet to pull the information about a landmark from Wikipedia that you are trying to shoot.

Probably every participant in the Expo was using Twitter or Facebook. Alex Payne of Twitter explained why they have chosen Scala to solve scalability problem in their backend queuing system. He said because of Scala they have an API test suite that helps them sleep.

User Interaction design grabbed good attention during a demo of Adobe Flex Catalyst in a keynote session that demonstrated how graphic design can be converted in to a Flex application with some cool transition effects using Catalyst. An interesting session was on user experience and software development Can’t We Just All Get Along? Human-centered Design Meets Agile Development by Maria Giudice who moderated a panel of four other people having software development and user experience design background. The panel covered the challenges in combining the short iterative nature of Agile projects with creative user experience design activity that creates conflict.

As the funding for new ventures dries up there is a remarkable increased in Agile awareness and adoption in the Web 2.0 community. Eric Ries, who is a popular author and speaker and publishes a blog titled Start up Lessons Learned, delivered a talk titled Lean Startup. In a jam packed hall, he suggested that a lean startup company should quickly iterate from ideas to code to data collection. Eric Ries ideas encapsulate various Agile practices in an iterative cycle and provide an overall big picture that many popular Agile methodologies seem to be lacking.

Some of the key take aways from this events are as follows:
1. Agile is at the point of becoming mainstream way of software development
2. Mobile Internet and Geo Location APIs will drive application development in coming years
3. User experience and new ways of Human Computer Interaction will drive application development as demonstrated by Wii and iPhone
4. Value of social network grows geometrically as number of users increase
5. Web 2.0 technologies are making their inroads behind Enterprise firewall
6. Public API makes integrating content easier which amplifies the network effect of a social platform

Adobe Systems, IBM, Salesforce.com, Nokia and Microsoft were the major vendors who participated in this event as exhibitors.

Keynote presentations including very inspiring lectures from Tim O’Reilly and John Maeda are available on blip.tv.