I am addicted to a problem: handling unknown peak load on the net. Part of the solution I have in mind involves, of course, a fast web-server. One of the fastest around is Spray-can (see https://github.com/spray/spray-can) and I really like the thing for several reasons I won’t explain here. Anyway, I’m sure you can guess my very first question by now:

How fast is Spray-can really?

So I’m after assessing the speed of Spray-can. I can believe the guys from spray.io who tweeted (my co-worker Raymond Roestenburg pointed this out to me, thanks!):

Screen shot 2013-08-02 at 1.52.25 PM

Needless to say just believing these guys is absolutely no fun. Here is what I did to figure it out myself.

The Server.
I wrote this bit of scala to obtain a response that is easy for counting:

   case HttpRequest(GET, "/dispatcher", _, _, _) =>
      counter = counter + 1;
      sender ! HttpResponse(entity = counter.toString())

Let’s see if this works. It does! First request (http://localhost:8080/dispatcher) gives ‘1’, next gives ‘2’, then ‘3’. Cool!

The Client
Over the last months I have used several techniques for the Client. I started with JMeter and I blew up JMeter, not Spray-can. Then I wrote a really mean low level client in java, used thousands of threads and got results that I still do not understand. I might get into that in a later blog. Last tuesday I told my co-worker Joris de Winne and he asked why don’t you just use the ‘wget’ Unix command. So we hacked up this experiment the same day.

First experiment: one Mac

On my Mac there is no ‘wget’, so we used ‘curl’ but that’s a detail. We used two little shell scripts the first one (“testit.sh”) does curl just calls the server. It looks like this:

#!/bin/sh
curl http://localhost:8080/dispatcher 2>&1 > /dev/null

And a second one for making our live easy. I put the first script in an endless loop and start the loop 30 times in the background. Like this:

#!/bin/sh
while [ "" = "" ] ; do ./testit.sh 2>&1 > /dev/null; done &
while [ "" = "" ] ; do ./testit.sh 2>&1 > /dev/null; done &
while [ "" = "" ] ; do ./testit.sh 2>&1 > /dev/null; done &
<snip>
while [ "" = "" ] ; do ./testit.sh 2>&1 > /dev/null; done &
while [ "" = "" ] ; do ./testit.sh 2>&1 > /dev/null; done &

The mysterious result

Note I’m running Spray-can as well as my test scripts on the same machine (2Ghz Intel core i7)
This is what a see. A repeating pattern that shows a CPU bound process that drops frequently to almost no CPU usage at all.... Green is CPU power used by user processes, Red is CPU power used by the system.

Screen shot 2013-08-02 at 10.57.45 AM

But why this pattern???? I hooked up JConsole to see if this was garbage collection firing. Nope! I could really use your help here. Are my Akka Actors collapsing and being put put back in the air by supervising Actors???? And it gets more mysterious in a bit when we run our second experiment. Hang in.

Throughput

I let my experiment run for 5 minutes and then used a browser to see how many requests were handled. There were 161447 requests. So that is a throughput of about 538 req/sec.

Second experiment: two Macs

Today I used my wife’s Mac to run the clients on. It is a somewhat older machine and not as powerful (2.53 Ghz Intel Core 2 Duo) compared to my own. As we now have two machines there is obviously a network in between and I got myself some UTP cables to make the communication as fast as possible. That appeared unnecessary! Just using the wifi I saw this:

Mac running the clients:

Schermafbeelding 2013-08-02 om 13.04.24

Mac running Spray-can:

Screen shot 2013-08-02 at 1.04.14 PM

The machine running the clients is clearly CPU bound like we saw before. And the machine running Spray-can behaves as expected. But where is my repeating pattern now? I haven’t the foggiest....

Throughput

I saw 44697 requests in 5 min which is about 150 req/sec.

So now what?

I plan to organize a “Please Break My System” session with all my co-workers. I’ll allow any technique used for the clients except that they have to use their laptops as using servers in the cloud is no fun. Watch this space.