Acceleration is a must

Scrum advocates the use of pokercards quoting the estimation of work items in story points. Velocity is the total number of story points a team can take on in a sprint.

Next to using velocity for planning, velocity is often promoted as a measure for improvement. Learning, solving impediments, implementing improvements, more fun and better team play all should lead to an increased velocity over sprints. Steady or even declining velocity is a signal of a mediocre, non-improving team.

The theory and actual practice of estimations in story points diverge, which makes drawing performance conclusions out of velocity records a tricky business. The question is whether velocity should be used as a measure of improvement at all. This blog explains why you should not, or at least that doing so should be reserved ‘for trained professionals use only’.

How do you like your Story Points?

In a last-summer blog (June 21 2010, its-effort-not-complexity) Mike Cohn qualifies the relabeling of story point to complexity points as ‘wrong’. He states: “Story points are not about the complexity of developing a feature; they are about the effort required to develop a feature.”

If we follow Mike, story points are a function of complexity, amount of work (‘sheer volume of work’) and expertise (of the team on the domain). Maybe uncertainty should be part of the formula as well. The consequence of this definition is that the amount of work reduces when the team gains more expertise, uses smarter solutions or can profit from structural improvements.

So following Freyr in his first comment on Mike’s blog “a story last year which was a 5, now has some process improvements and this year is a 2. The velocity of the team stays steady, story points per story decrease.” Let’s call this method A.

This way to determine story points differs from the way I was raised with: user stories are estimated relative to a fixed (set of) reference user stories. A story of 5 last year is still a 5. Implemented improvements (whether technical or on teamwork, skills, knowledge or competences) should lead to higher performance so that we can do more such stories in one sprint. Velocity increases, story points per story stay steady. Let’s call this method B.

Count on Story Points while planning

In Scrum estimations and velocity recordings are primarily used for planning purposes. User stories are estimated in story points, and the number of stories planned in sprint depends on the sum of story points of these stories and the velocity of the last sprint or two. Furthermore, story points and velocity records can likewise be used for release planning.

Whether using method A or B a team can properly predict the result of a sprint. And in both cases one can calculate and forecast the number of sprints needed to finish a selected set of user stories in a release.

However, using method A the team will need to deal with new competences leading to lower estimates, even for user stories estimated in ‘the past’. Using method B the team needs to deal with improving velocities when predicting the number of sprints needed for a set Release.

Measuring improvement is a different story

Scrum promises an increase in productivity / performance. A growth factor of 4 even up to 10 is often quoted. Soon after the introduction of Scrum management requests proof of this growth, especially after they have been baffled with all formerly hidden impediments in their organisation: Scrum does not solve your problems, but reveals them! Putting a heavy burden on the belief in reaching the benefits promised.

Very often velocity is used to measure this improvement. This is only valid if we keep the story points for the same kind of stories steady, as shown in method B. In fact this should only be done if we restrict the formula of estimating story points to size, the size of a piece of work. This is shown by an example by Jose in another comment on Mike’s blog:

Imagine that we have a team of painters and their job is to paint walls. The team sees pictures of the walls and they estimate the wall area to be painted. There are small walls, medium walls and some really big walls. The team estimates each area using relative points. They start painting the walls and after some time, they are able to calculate how many points they can do on each sprint, the velocity. Now the customer can calculate how long the project will take, using the velocity and the product backlog.

The problem is that determining size is one of the most difficult challenges in the field of software development. There certainly are environments where this works fine. Recently we coached a team in a Business Intelligence environment, who were able to define their workitems in predefined steps, and who had derived formula’s from experience to calculate the size of work for each step. Their estimations were remarkably accurate.As soon as the painters get better tools and paint, or gain in experience and concentration they will be able to color bigger walls in the same amount of time.

For most other teams the thing coming closest is Function Point Analysis, which is far from easy and a not a common competence in Development Teammembers.

About every other team we have seen estimates effort. Better tooling, a higher grade of automation, more competences, all lead to less effort and are discounted in the final story points. Fine for planning, but using velocity trends based on these story points is a very tricky business and will not reflect the growth teams realized to the amount they deserve.

The point of this story: size matters

Story points are calculated using different methods and formulas. If your formula does not equal just sheer size, you should not use your velocity measures as an indicator for performance growth. Like you cannot learn your performance development by simply comparing the scores of two games of rugby. If there are many other factors influencing the scores you will do wrong to the team and you should stick to use these records ‘to start a conversation’ only. Measuring the improvement in performance ain’t easy.

From their homepages:

Node.js: Evented I/O for V8 JavaScript. (A framework for building completely non-blocking servers in Javascript)
NPM: A package manager for node.
CoffeeScript: A little language that compiles into JavaScript
Express: High performance, high class web development for Node.js
Jade: Node Template Engine
Redis: An open source, advanced key-value store

In this guide I will take very small steps so that you can verify that you are check whether you are still on track.
The result is an extremely performant, scalable and lightweight alternative for web development.

So I installed a brand new Ubuntu 11.4 virtual machine and got started. After following some tutorials, googling and some tweaking this is the step-by-step guide I came up with.

Installing Node.js

$ sudo apt-get install python-software-properties
$ sudo add-apt-repository ppa:jerome-etienne/neoip
$ sudo apt-get update
$ sudo apt-get install nodejs


Installing npm

$ apt-get install curl
$ curl http://npmjs.org/install.sh | sudo sh


Node.js Hello World

$ mkdir helloworld
$ cd helloworld
$ vi helloworld.js


insert

var http = require('http');
http.createServer(function (req, res) {
  res.writeHead(200, {'Content-Type': 'text/plain'});
  res.end('Hello World\n');
}).listen(1337);
console.log('Server running at http://127.0.0.1:1337/');

$ node helloworld.js

check http://127.0.0.1:1337

Add Coffeescript

Writing all that javascript is going to lead to lots and lots of brackets, semi-colons and parantheses. Let’s use Coffeescript to make our code readable again.

$ sudo npm install -g coffee-script
$ cd ..
$ mkdir hellocoffee
$ cd hellocoffee
$ vi hellocoffee.coffee


insert

http = require "http"
http.createServer( (req, res) ->
  res.writeHead 200, {"Content-Type": "text/plain"}
  res.end "Hello Coffee!!"
).listen 1337

console.log 'Server running at http://127.0.0.1:1337/'

$ coffee -c hellocoffee.coffee
$ node hellocoffee.js

And check http://127.0.0.1:1337 again.

Adding Express

Node.js is a very powerful, but it still requires quite a bit of boilerplating to use it for web development. That’s where Express comes in. It is build on top of another middleware layer called Connect and gives you much better support in handling requests, rendering, templates and responses.

$ cd ..
$ sudo npm install -g mime qs connect express
$ mkdir helloexpress
$ cd helloexpress
$ express
$ npm install -d
$ node app.js


check http://127.0.0.1:3000

vi app2.coffee

insert:

express = require('express')app = express.createServer()

# Setup configuration
app.use express.static(__dirname + '/public')
app.set 'view engine', 'jade'

# App Routes
app.get '/', (request, response) ->
  response.render 'index', { title: 'Express with Coffee' }

# Listen
app.listen 3000
console.log "Express server listening on port %d", app.address().port

$ coffee -c app2.coffee
$ node app2.js


And check http://127.0.0.1:3000 again.
Now that we have Coffeescript working with Express let’s add sessions.

Add session support

$ vi app2.coffee

insert after app.use express.static(__dirname + ‘/public’)

app.use express.cookieParser()
app.use express.session {secret: "Coffeebreak" }

insert after app.get ‘/’, (request, response) ->

request.session.views++

replace

response.render 'index', { title: request.session.views + ': Express with Coffee and sessions' }

$ coffee -c app2.coffee
$ node app2.js

Check http://127.0.0.1:3000 and refresh a few times. You should see the counter increase.

Install Redis

Open a new terminal.

$ wget http://redis.googlecode.com/files/redis-2.2.11.tar.gz
$ tar -zxvf redis-2.2.11.tar.gz
$ cd redis-2.2.11/
$ sudo apt-get install build-essential
$ make
$ ./src/redis-server


Add redis session to node

$ npm install -d redis connect-redis
vi app2.coffee

insert after express = require ‘express’

RedisStore = require('connect-redis')(express)

change

app.use express.session {secret: "Coffeebreak", store: new RedisStore, cookie: { maxAge: 60000 } }

$ coffee -c app2.coffee
$ node app2.js


In the Redis terminal you should now see something like:
[25489] 24 Jun 04:57:11 - DB 0: 1 keys (1 volatile) in 4 slots HT.
[25489] 24 Jun 04:57:11 - 1 clients connected (0 slaves), 799032 bytes in use


Check http://127.0.0.1:3000 and refresh a bunch of times
kill and restart node

$ node app2.js

Check http://127.0.0.1:3000 again and refresh a few more times
Note that you started where you left off.

And that brings us to the end of this guide. As you can see the stack is very powerful and extremely small, lightweight and clean. With all the session information in the Redis store the availability of the entire systems comes down to the availability of Redis. And with these guys working on a proper clustering solution it will be very interesting to see where all this is going in the near future.

Around 2006 several initiatives to create open source performance monitoring tools for java production environments started to appear.

This was mainly because AOP (Aspect Oriented Programming), the technology used in most of these products, was getting attention in the market and there were quite some developments in that area at the time.

I am interested to see how the open source community around these kind of products is evolving. The outcome is quite surprising…

The main reason I am interested in tools in this area is to be able to pinpoint problems in an Acceptance or Production environments very quickly, because this saves (big) enterprises a lot of money.

The main reason I am interested in open source tools is to see if we are able to create a complete open source java hosting stack which is enterprise production ready.

Without adding a lot of extra license and/or support costs we don’t have to consolidate IT anymore. We will be able to do application hosting for each specific business unit separately and focus on the continuous improvement for that business unit alone. Without big boundaries between the business unit and IT. This kind of boundaries are most times created because of cost efficiency. Think about consolidation and architecture guidelines. Not having to think about licencing costs will help us in helping enterprises to become agile.

During a knowledge exchange session (XKE) at Xebia we first created a requirements list to select tools in the same area to be able to compare them.

The requirements so far:

1. The tool should measure production and acceptance environments with low (<= 5%) resource usage overhead.
2. The tool should be able to pinpoint (potential) problems in the running applications as quickly as possible.
3. The problems that should be detected: high resource consumption of particular code blocks, memory leaks and the location of the leak, blocking code blocks which block other threads, slow calls to back ends (i.e. web services, databases).
4. The environments the tools should run in are the following application servers: IBM Websphere, Oracle Weblogic, JBoss, Tomcat, Glassfish.
5. The problems should be made visible in such a way that the root cause can be found as quickly as possible. The most valuable will be to see complete business transaction flows over all JVM’s, with real-time flow states and measurement readings.
6. The tool should be easy to install, i.g. just use a simple agent that connects it to a server. The goal should be to leave the code base unchanged.

Before the session I did some homework and I searched the internet for these kind of monitoring tools. During our knowledge exchange session we had a brainstorm session and came to the following list of Open Source tools:

This list is as complete as we were able to come up with, if you know more and eventual better open source tools please let us know!

After evaluating all products on the lists (just a paper evaluation) we only see Glassbox and Infrared as a real Open Source solution in this area. But both (at least Glassbox) are not actively developed anymore. Of course because both products are open source it will be possible to restart development on one of the products.

It is interesting why both projects are not active anymore. I spent some time to find out why. This is what I found out:

What happened to Glassbox; the first developer mentioned is the VP of Engineering at Glassbox David Pickering. He has left the Glassbox company in 2009. And is now VP of Engineering at iWin, a company with a different focus. The second one mentioned is Ron Bodkin, he left Glassbox in 2007 and was the founder of the Glassbox company, he became VP of Engineering at Quantcast, so also his focus has moved. This looks like the main reason there is no active development on Glassbox anymore.

What happened to Infrared; the first developer mentioned on the project member list at SourceForge is Binil Thomas, he worked for Tavant Technologies at the time Infrared was created. Other members on the same list also worked for Tavant in that time. Most are now working for different companies. It looks that Tavant Technologies was no longer interested in this space. The last 2.6RC1 release from Infrared was created in 2010, at a time Binil was working for Ronin Capital. Since 2011 he is working for AppDynamics, this is a commercial vendor in this area and looks like the main reason the 2.6RC1 release never came to a full release, but this is something I am guessing of course.

What’s next

We will test both Infrared and Glassbox to see if one of them is still a good choice. We will start with Infrared because this one has been the most active project in the past and it has been maintained the longest, until 2010. After this we will see if one of these products can be a competitor for the commercial products in this area.

Many thanks to my colleagues Adriaan Thomes and Sander Hautvast for helping me with the brainstorm session.

We are sure the Agile hot topics for 2011 are:

Agile Management

The managers finally find their place within an agile organization. In 2010 a lot of managers struggled with their position, in 2011 they find their spot. It’s called facilitation island and it smells like paradise and still feels like hard work.

Agile Management Consultancy

The management consultancy world invites itself to the party. The consultancies bring new (project)managementmodels around Agile with them to establish their position in the agile world. This is a mixed blessing: it’s both confusing and useful at the same time. Confusing because Agile is put in models that oppose the Agile values and put more emphasis on process tracking, Big Up Front Plans and management control. Useful as it attracts senior management attention to Agile and help accelarate agile thinking on typical board management topics as Agile strategy implementation, Agile KPI management, Agile risk management, Agile budget control and Agile organization structures.

Agile at Scale

Corporations shift from scaling agile to agile at scale. Instead of using Agile only for projects, agile is implemented at corporate level with a stronger focus on product portfolio management to deliver more value. Senior management gets seriously involved in the prioritisation of what’s really important. The discussion is no longer about prioritizing projects, it’s about what delivers most value to the company in the long run.

Agile Maturity

Many companies start their journey with agile. They seek to learn from the past experiences of those that went before them. Those trailblazers on the other hand are ready for the ‘beyond Agile’. Agile maturity with a focus on the journey from just starting to completely incorporating agile is the subject of many publications and presentations in 2011. Agile maturity is how the different groups share their experiences and learn from each other.

Agile Metrics

Now the use of Agile has become more and more serious, the need for good metrics has become even more real. The Agile world rids itself of its cowboy stigma by creating more transparancy in a way that is useful for everyone in an organization. Putting things on a wall is no longer enough, you can’t expect everybody in a 30,000 people company to visit all the teams daily or even weekly. The big agile adoptions embrace metrics to enable running agile at scale.

Risk Management

Given these other trends risk management becomes more important. The IT risk management so far has been the laughing stock of real risk management. 2011 is the year IT risk management matures and agile leads the way.

Agile Results Only Work Spheres

The Results Only Work Environment (ROWE) or ‘new way of working’ (Dutch: “Het nieuwe werken”) is here and implemented by many corporations. While corporations want the benefits from working with Agile, there is also a strong desire to apply ROWE. An answer is found to combine distributed working and teameffort into a new way of working called: Agile Result Only Work Spheres also known as AROWS.

We wish you all a pleasant 2011 and may it be a MoreAgile year !

In this growing on line world with demanding customers it has become essential that services provided on the web are always available and always fast enough. This is often challenging to developers and operators: performance problems manifest themselves in various ways, like in frustration, loss of revenue and disruption of development; and just adding hardware is a doubtful solution.

The question is: how can we as developers and operators assure that our web site is always available and always fast? My answer is: you need the right approach. I present that approach: measure, don’t guess; seven steps to performance success. These seven steps are as follows:

• Step 1: Define performance requirements;
• Step 2: Execute a proof of concept;
• Step 3: Test representatively;
• Step 4: Test continuously;
• Step 5: Monitor and diagnose;
• Step 6: Tune based on evidence;
• Step 7: Share the responsibility for the whole chain.

This approach provides a pro-active way of working which my customers appreciate as valuable. It can actually be leveraged to assure high performance all the time, for virtually any on- and off-line application.

This blog series has been an interesting journey for me. Some time ago we presented our EJAPP Top 10 of performance problems. Now we have added this approach of seven steps to help assure your applications performance.

It has worked for us and our customers. How does this all work for you in practice? We’d like to hear your feedback.

When an incident happens in production, this usually means stress. A performance problem in production often leads to finger pointing. The DBA says that he has looked and nothing is wrong with his database. The network operator concludes the same thing about his network. The app server operator about his app server, the software developer about his source code and the back end operator about his back end. It is never them, it is the other.

Figure 1. Finger pointing does not help to solve the problem.

The application often gets thrown over the wall to the operation department. Responsibilities then hold only at one side of that wall. If software development, maintenance, testing and/or operation is outsourced to external parties this can lead to tricky situations. Before you know it, contracts and legal procedures are at play and cooperation is far away. Both parties stick to their position, costs will raise and precious time gets lost.

Finding out which part of the chain is responsible for the slowness can partly be solved with proper tools that monitor the whole chain and tools which are used from early on in the development process. But there is more to it than just tooling. Experience with and knowledge of tooling and technology is inevitable just as priority for the proper utilization of the tools. It is most important to prevent formation of separate kingdoms and finger pointing between them; and rather to operate together as a multi-disciplined performance team and share the responsibility for the whole chain.

We are almost there with this series Next time I’ll draw the final conclusions of these seven steps.

If an application turns out to be too slow, tuning can provide a solution. Tuning can take place on multiple levels. Adding hardware can be a cheap solution. However, when you add hardware at a place where the bottleneck is not located, this has little use.

Important steps of tuning are therefore identifying which pages or services do not meet stated requirements and isolating the problem: where is it located, in which layer, in which component. This can be made clear with testing and monitoring on application parts. The next step is diagnosing. In fact, this comes down to making up a hypothesis why this component is so slow.

Figure 1. Performance tune cycle.

This can for instance be a missing or wrong index on a database table or the invocation of too many small queries. Next, the component is improved based on this hypothesis. Finally, one needs to verify whether the improvement actually brings the expected speedup. If so, then the proposed hypothesis is true and the speedup is the result. If not, then there is something wrong with the hypothesis and we need an alternative hypothesis. As soon as the performance of the system meets its requirements, tuning is finished.

Figure 2. Finding evidence.

The right tools are indispensable: performance test tool, enterprise profiler, heap monitor, etc. I have seen developers work on assumed performance improvements which turned out not to help at all, or even slowed down the application and also deteriorate the maintainability and flexibility. This is caused by the fact that developers are used to mould functionality from source code and therefore work from source code to improve performance. What is missing here is measure, don’t guess. This is something developers learn in my performance training. Experience also has taught me to judge every proposed improvement separately and to only implement the improvement when we have proven that it really helps. Without this systematic approach, a handful of steps together may take you backwards at the end of the day instead of forwards.

Next time I’ll talk about Sharing responsibility for the whole chain.

When a new version of the software is released into the production environment, the question always is: will it actually perform like we saw in testing and acceptance environments? And we keep our fingers crossed.
It is therefore important in such cases to monitor carefully what happens with the performance and availability of the application.

There are all sorts of tools and services available to monitor your web site for availability and response times of web pages, like Uptrends, Site24x7 and Dotcom-monitor. They look at the application as a black box and measure once in several minutes. This is very useful, however, to be able to take the right measures in case of a calamity, it is necessary to be able to pin-point the problem. It is therefore essential to monitor on multiple levels and on multiple internal application parts. For levels, think of hardware, OS, app server, web server, database and application. Measuring internal Java application parts can be achieved with JAMon. JAMon is an open source timing API and basically works like a stopwatch with a start() and stop() call. Every method which you want to measure gets its own stopwatch (or counter) . We deal with JAMon as one of the tools to measure time in the first day of our Speeding up Java Applications course.

Figure: JAMon API start() and stop() calls in a Spring interceptor

Each counter maintains statistics like the number of calls, average, maximum, standard deviation, etc. , and this information can be requested for. The individual calls are not stored. This approach results in low memory usage and a low performance overhead, at the cost of some information loss. Recently, a new competitor of JAMon appeared: Simon. It claims to be JAMon’s successor, although it has (had) some infancy issues.

Then there is the question: where to measure? The answer is that it makes most sense to measure all incoming calls like web requests and outgoing calls to for instance the database. Furthermore, parts like Spring beans, EJB’s and DAO’s. Measuring these parts is not only relevant with new releases, but also trends and usage spikes are useful to monitor in order to solve quickly and prevent various problems. Open source tool JARep offers the possibility to store JAMon data from a cluster in a database and monitor trends and changes graphically.

Figure: JARep shows the increasing response time trend starting October 15, on two of the four production JVMs.

Customer story

We had the following situation at our customer. Processing an order slowly took more and more time over a period of several weeks. This happened while no new release was introduced and no other page became slower. This behavior was a complete mystery, until we looked deeper in our JARep monitoring tool. The troublemaker turned out to be a DAO executing a prepared statement with only part of the variables being bind-variables. With help of JARep, we could look back to where the trend of increasing response time started so when the problems started. We could also see that this problem was only present at one of the two machines. With this knowledge and his log book, the operator could remember that on the start date he had experimented with a new JDBC driver to try to solve a memory leak. This seemed not to change anything concerning performance, what actually was the case in the beginning. Problems only appeared slowly during the following weeks. They had left the new driver in place, manifesting itself as a time bomb later. When we put back the old driver, the problem disappeared. This real life experience shows the usefulness of monitoring and trend analyses on application internals.

Next time I’ll blog about evidence based tuning.

Continuous performance testing
With a representative test as one of the last steps before going live we prevent that expensive bad-performance surprises will pop up in production. However, the same surprises will pop-up, only earlier and with less impact. To save costs and prevent large architectural refactoring, it is crucial to test for performance as soon as possible. This is just like any other software defects and Quality Assurance: the later in the development process defects are detected, the more costly these defects are.

At a popular web shop I had the following challenge: we wrote the performance tests only at the end of the six-weekly release period, after functional testing had taken place and functional defects were corrected. In case serious performance defects popped up, a crisis team was gathered and we found ourselves in a stressful situation. There was usually not enough time to fix the defect before the release date, so my recommendation at times was to defer the release date. However, deferring the release date often just was not possible, because TV or radio time was bought to promote the new functionality. So, how to solve this dilemma? We found the solution in applying agile principles: test features as early as possible and the team is responsible.

We included meeting performance requirements in the definition of done in each new or changed feature individually. The development process already included an automatic build, quite common these days. Unit tests of a feature were written as usual by the developer. We now introduced performance tests to the spectrum: the developer writes the performance test script for his feature (service or web page) in JMeter, side-by-side to his unit tests on the classes. When the nightly build with Maven has taken place, the application is deployed on WebSphere and the performance tests are run by the JMeter Ant script. This script generates a report which is e-mailed to stakeholders. In this way, the IT department gets early insight into new and changed features, it can adapt its course more quickly, back-off early from an unfortunate architecture or approach, minimize surprises and have lower costs as well. Additional benefit is that writing test scripts gets done more quickly than before, because the developer has all details of the new feature still fresh in his memory. These details are for instance the conditions under which the service may be called and by which parameters, in which variations and special cases. The usual communication overhead between a performance tester and a developer on these details is drastically reduced, thereby further improving productivity.

Continuous performance testing is too often underestimated and un recognized in my opinion, it really has a whole lot of advantages and it is not that hard to achieve.

Next time I’ll blog about Step 5: Monitoring and diagnostics.

Slowness of applications in development environments is often neglected with the rationale that faster hardware in the production environment will solve this problem. However, whether this is really true can only be predicted with a test on a representative environment and in a representative way. In such an environment, there needs to be more representative than just the hardware.

I have experienced multiple times that a database query on the test database with 1000 customers took only less than 10 ms., while on the production database with 100.000 customers this turned out to take tens of seconds, because of missing indexes. So, if the development team does not test with a full, complete database, going to production may lead to some surprises.

It is also important that the number of concurrent users and their behavior is well simulated in the test. Furthermore, care should be taken to take caching effects into account: if the test continuously requests for the same product by the same customer, this data will be in database or query cache the second and following times. This will speed up the request considerably and be much faster than with many customers and products. This test is therefore not representative for the real situation.

A suitable performance test tool and performance expertise is necessary to create a valuable test. The most popular open source performance test tool is Apache JMeter, see the next figure.

Figure: Screenshot of a run of a performance test in Apache JMeter.

We deal with this tool, and how to performance test properly in our Speeding up Java Applications course. This is a tool made by programmers, for programmers. Test scripts can be created with visual elements like a HTTP request, which can be recorded and configured. Many are available and if you need more, you can always fall back on a BeanShell element in which you can manipulate the request, response and various JMeter variables. If that even does not meet your needs yet, you can extend JMeter source code and develop your own elements. Because of its for-programmers nature, it is less suited for the average tester. Also reporting features and maintainability of the scripts are both not so great. Therefore, commercial tools like HP Mercury LoadRunner, Borland SilkPerformer or Neotys’ Neoload may be good alternatives for companies.

Performance testing from the cloud

The emergence of cloud computing adds new possibilities for performance testing. An elastic compute cloud like Amazon EC2 provides the ability to scale up quickly with the number of application deployments because of increasing load. For performance testing, the cloud can be used the other way around: for temporary use of many load generating test clients to generate expected and peak loads for your application. This saves you from having to buy many servers to run the load generating clients and if you run these performance tests say only a couple of days in a release cycle, this can be an economically attractive solution. Quite some information is available how to test with various performance tools from the cloud.

Next time I’ll blog about step 4: continuous performance testing.