This year I had the honor of presenting a case study on Fully Distributed Scrum together with Jeff Sutherland, co-founder of Scrum.

We presented about a Xebia client located in San Francisco working with our office in new Delhi using a single hyperproductive distributed Scrum team! Thats right, hardcore Agile results across all timezones, culture, language etc.

In previous work Jeff and I have explained the secret sauce of Fully Distributed Scrum and explained the repeatable recipe. However, those case studies used the window of time overlap between Europe and India to have most communications.

The United States and India do not have this luxury. The time difference between San Francisco and New Delhi is absolute. There is 12.5 hours difference, so no matter where you go on this planet, it will not get any worse then that!

What is Fully Distributed Scrum?

Offshoring classically is performed in a waterfall process fashion, meaning that one party writes specifications and chucks them over the wall to the other party who then chucks a software product back after a period of time. When evolved to Agile software development, this is the last thing you want to do. However you do need to deal with the distance between your local staff and your talent in an offshore location.

The only way to get the benefits of both Agile development (hyperproductivity with high quality and motivated people) and the benefits of offshoring (lower costs, availability of talent, up/downscaling with no risk) is to apply Agile to the offshoring dilemma. Xebia has made this the core practice and differentiator of our offshored development.

At Xebia we create teams with engineers on multiple locations. These run Scrum Sprints where the teammembers on both locations share a single sprint backlog and a single sprint goal. The entire team is responsible for the result and there is a very flat hierarchy and a sense of equality.

This mechanism is called Fully Distributed Scrum.

The case study

TBD.com, a social networking company, is located in San Francisco. For a period of 8 months they have extended their staff with Xebia India working in a Fully Distributed Scrum fashion. In order to get this fully working we have had to make a number of adjustments to the Scrum cycle. Over the period that we worked together the business success factors (number of members and activity per member) quadrupled!

Have a look at the whitepaper and the presentation for all the details on how to make such an engagement work.

Download article    Download presentation

The room was great, about 75 enthusiastic people with some very sharp questions.

Turns out that although in general people tend to agree what a cloud is, nobody actually knows exactly what to do with it!

Lets start with a little overview of Everything cloud:

• IaaS
• PaaS
• SaaS

Confused yet?

IaaS: Infrastructure as a Service. The obvious and always used example! This is all about virtualization. You don’t want to know about the underlying hardware anymore, you just want to be able to launch a new instance of your favorite operating system using a configuration tool. Amazon EC2 is the default example for running the OS and Amazon S3 is the default example for storage.

PaaS: Platform as a Service. This is basically the concept that you follow some (proprietary) rules to write your application so that it can run on somebody elses cloud and you dont want to know anymore about any underlying infrastructure, hardware, operating systems, app container, database management, etc! Examples are Google App Engine and Force.com. And unfortunately CogHead.

SaaS: A bit of a strange item in this list. It means Software as a Service. You use software (usually webbased) that is offered to you by a third party on their systems. And their systems just might be running on a cloud like platform, you just don’t care. There is lots to configure and lots to personalize in main examples of this like SalesForce. It smells a lot like ASP (Application Service Providers) and often has a fair bit of Portal mixed in. No radical new concepts.

Turns out that the whole hype around cloud does not come from radical new technologies, it comes from the different pricing models and a bit from separation of concerns. The most used arguments for using cloud stuff are based on an IaaS model: Instantly available hardware, pay only for the cycles that you use, don’t bother anymore with maintenance of the physical stuff, and scale up for bursts like end-of-year calculations. Unfortunately, since pricing models are currently based on this bursting concept, it is not fully applicable to a lot of our businesses.

An interesting way to look at these different models is that if you move down the list you get a lot more freedom from concerns. At PaaS you already do not worry about everything connected to your application logic. You have a container interface that is provided and you just do not want to know about the rest. This could help speed up your development a lot. Possibly. Perhaps. The cost associated is that you are very proprietary and tied in to you provider!

This tie in might not seem so bad untill you look at the example of CogHead. This PaaS platform was bought up by SAP recently. However SAP did not buy the clients and did not continue the public service. That means that these people have received a legal notice that they have 3 weeks to move off the platform before service is discontinued. Now, can you rewrite your entire business in 3 weeks? And handle a migration? Probably not!

Where SaaS obviously has a similar problem, the IaaS models seems safe enough. You do everything the way you want, probably creating images with the usual platforms that you can get anywhere, also in non cloud environments. The most risk here is your data. There are a number of examples of people with a client base that now have so much data going into Amazon S3 that there is no way to get it out again in a reasonable timeframe. And then there is the safety of data and regulations which is creating the discussion about public and private clouds (shared usage or just for your company) and hybrid models between your infra and the cloud.

The issue that this raises and what we discussed most at the Enterprise Cloud discussion at CloudCamp is that the problem currently with Clouds is not the technology (although PaaS is just beginning), it is mainly the SLA’s and Escrow, pricing and contracting models that are not mature. It is developments in this area that will drive innovation and usage, not the next technical scalability solution.

How to get a focus on individuals and interactions when your people are distributed across the globe? What is the secret sauce to use to get it running smoothly?

The classical route of bringing this ‘gap’ under control involves adding process and handovers. It actually forces you to go into a waterfall-like model and therefor widens the Gap instead of bridging it. All waste is institutionalized. Sounds like a horror to you? It does to me.

Agile is the key to radically eliminate this waste and get the full power of both Agile and offshore benefits. However Agile and offshoring seem like oil and water, at first glance they don’t seem to mix.

What I talk about at QCon is just how to blend the two. I’ll discuss the different models that in use for distributed Agile along with their impact. The final solution being a fully distributed scrum model that allows for linear scalability and causes the team to behave like a single local team in most aspects.

To just tell you what it is all about: People. The issue with distributed development is indeed the distance but measured in people, not in miles. The Fully Distributed Scrum is the approach that we use very successfully to deal with that distance.

The case story from the presentation is explained in much more detail in the paper that Jeff Sutherland and which we wrote after a lot of great experiences at Xebia.

The presentation was filmed and will be online in due time with the great folks at InfoQ.com. Untill then:

Download the presentation slides or Download the original paper

Agile 2008

At Agile2008 in Toronto Jeff Sutherland and myself presented our article outlining how to achieve hyperproductivity in distributed Scrum when working in an offshore situation. InfoQ recorded our presentation and will publish it online in November as the end of a series of Agile2008 talks.

Download article    Download presentation

Also see this InfoQ article

Agile and Offshoring, oil and water?

If you are reading the Xebia blog chances are that you are already familiar with the benefits of Agile development. Practicing Agile (in our case Scrum combined with XP) delivers hyperproductivity combined with very high quality. The promise of offshoring in the modern IT industry is also clear: more available talent, scaling up and down without local layoffs or knowledge drain, and of course cost reduction. Together they make a killer combo!

However, Agile and offshoring seem like oil and water, they don’t seem to mix. How to get a focus on individuals and interactions when your people are distributed across the globe? What is the secret sauce to use to get it running smoothly?

Distributed Outsourcing Styles

There are different models that you can consider when distributing Scrum. The most primitive model that is often applied is to create fully separate Scrum teams that work on unrelated work. Slightly more advanced is the usage of isolated Scrum teams that connect regularly via a distributed Scrum of Scrums. This is the model that the Scrum Alliance currently advocates.

Much more effective is a Fully Distributed Scrum model where each team consists of members on both locations. In this model there is no division of teams based on geography, teams span the globe! At Xebia we came to this model simply by reasoning from the Agile manifesto. The main issue in offshoring is communication. When creating a geographical division between teams you create another communication barrier that you need to take. People tend to take the road of the least resistance and end up with all sorts of processes and tools.

When creating a distributed team, with members on both locations, the team members have no choice but to resolve their communication problems.

Once these are resolved you can operate on a basis of equality, providing your offshore team members with responsible intelligent work, thus giving them the job that they deserve as 21st century knowledge workers.

Shared mindspace

Setting the team up for success has a lot of it has to do with getting into a shared mindspace across locations. This means developing a shared ownership of the code and architecture, a shared domain context and a shared value system and culture. In order to set this up you need to think carefully about traveling, about building personal relationships, about finding the commonalities that Agile developers the world over share. Setting this up and cherishing it is vital to a smooth operation

The presentation

Presenting these concepts that I am so passionate about at a conference like Agile2008 is a fantastic experience. Distributed Agile is a topic that is obviously very much in the spotlight. The audience had a lot of questions and it was great to share ideas after the presentation.

When talking to the people that attended our presentation I noticed that a lot of people are in the process of training their offshore partner to become more Agile. Usually first through a Scrum of Scrums model and eventually aiming for a Fully Distributed Scrum. Setting it up in this way, by changing your suppliers habits, is a very slow and costly process.

Inspired by the success of projects like this one we are building distributed teams now for partners, with a match on Agile and quality. If you have an Agile business where you are looking to include the benefits of offshoring then we can help by hooking you up to Xebia India. Have a look at Xebia Global Services for more information.

Next step is JAOO in Aarhus, Denmark where Jeff and I will present about this topic.

Guido Schoonheim, CTO

Just a few highlights of a day filled with high quality talks:

A gripping keynote was delivered by Pete Deemer to kick off the day. Basically Pete warned us for the hardships of Agile adoption by discussing his top 19 lessons of Agile adoption at Yahoo! and other companies. While some companies experience a delight when implementing Agile others have a very hard and frustrating time and don’t reap the full benefits. His 19 lessons were geared towards making sure you fall into the positive category.

The point he discussed that stayed with me most was the one about management commitment. In many cases Agile is great, fun, fine and nice, but when a project reaches a critical phase and for some reason will miss the deadline then management still utters the three magic words…

MAKE IT HAPPEN!

This is the point where the team goes into ‘by all means’ mode. Quality and sustainability go out the window as adding resources to a late project is not a sane option. This is done both by skimming practices and by working overtime. Such a team leaves a wave of destruction in the form of technical debt in its wake. If they are unfortunate enough to make it then new ambitious deadlines are based on previous successes and it becomes very unlikely that the extreme cleanup operation needed will ever be undertaken. Thus creating a costly spiral out of control. Great thing about Pete is that he is such an engaging speaker that you can see depression settle over him as his narration of this scenario progresses, leaving the audience deeply impressed with the awfulness of this cardinal sin.

Naresh Jain had a good overview talk, explaining the mismatch between traditional engineering practices and software development, tying in Lean and queueing theory. An important note that he made was on the clarification of the Agile Manifesto Value “Working software over comprehensive documentation”. Documentation here does NOT mean (functional) user documentation. It means in between artifacts like design documents, architecture documentation, detailed up front requirements and other items that can be eliminated as waste.

Another highlight was Saket Vishal with his Distributed development case study from a developer perspective. Saket first asked the audience what they would do if standup meetings ran too long. When we all replied that timeboxing was a solution along with the phrase ‘take if offline!’ he had us where he wanted us. During the rest of the presentation he engaged the audience a lot and took us through the communication issues he was focussing on. However every time we got a bit fanatic in discussing the possible solutions Saket said ‘yes yes very well thank you, timebox is up, take it offline!’, cracking up the audience. A very well done and entertaining hour.

The openspace part resulted in a few very heated discussions. The one I partook in was Anurag Shrivastava‘s question: Does the Indian culture hinder the use of Agile development? I believe that all participants agreed that there were elements making the self organizing teams, intense direct communication and light touch hands-off management required difficult. No solutions were agreed upon, if we ever get that far we will certainly let you know.

We discussed the Agile adoption programs of various large Indian IT providers and did agree that adopting Agile practices because of customer demand without adapting your company value system to be compatible with Agile values is a long and painful road to failure. My opinion is that the apparent incompatibilities of Indian culture and the usage of Agile are a catch 22 that hangs on trust, and only improve the gains of Agile development in an Indian environment, but thats for another blog another time.

Finally Anurag kept everyone in total concentration with the lottery draw of feedback forms. A lucky winner could take home an iPod. Not only did Anurag manage to turn the announcement of the lucky winner into a show, he selected four possible winners and got us roaring by first reading out three members of the audience that got absolutely nothing.

All in all a great day that I was happy to attend. I only cover a few things that stood out, but enjoyed all the talks. It was very much a team effort, with more Xebians like Mayur and Deepak giving great talks and organizers like Anurag, Amit, Abishek, Jyoti and Kiran making it all work. I was impressed with the overall quality of the day and enjoyed the interactive audience greatly.

As we completely sold out this year I am sure we will be back next year, bigger and even better.

Guido.

Last week all of us at Inspiring visited the JavaPolis conference in Antwerp. JavaPolis is a five-day event organized by the Belgian JUG consisting of two university session days followed by three normal conference days. The university sessions were three-hour in-depth technical presentations. The fifty-minute conference sessions were less intensive but sometimes also less boring

We attended just two days, the last university day on Tuesday and the first conference day on Wednesday.

On Tuesday a couple of us visited the ‘Spring Business Tier’ session by Juergen Hoeller and Rob Harrop, who presented some new features of Spring 2.0.

The first part of their presentation was quite interesting, which covered the new XML namespace features. Instead of using the default DTD, in Spring 2.0 you can use XML schema’s in your configuration. By using schema’s it’s possible to add more ‘typing’ to your configuration, which reduces the chance of making mistakes. Besides that, I think it makes your configuration much more readable. Spring itself will have 4 default schema’s you can use, JNDI, AOP, UTIL and TX (Transactions) .

At the top of your configuration you can now declare your schema’s:

xmlns:aop="http://www.springframework.org/schema/aop"
xmlns:jndi="http://www.springframework.org/schema/jndi"
xmlns:tx="http://www.springframework.org/schema/tx"/>

And if you want to lookup a datasource through JNDI for example you can use a much more powerful and intuitive definition:



The whole idea behind this schema usage was to make the use of AOP through Spring much easier and the first step was to simplify the configuration.

This explanation lead to their second subject, improved integration with AspectJ. Spring 2.0 now uses the AspectJ syntax to specify pointcuts, even if you don’t use AspectJ as the AOP implementation. This makes working with AOP a lot more powerful. You can also use the same Aspects in Spring and in AspectJ without making any changes to your aspects. By using standard annotations (again…) your aspects will be compatible with both Spring and the AspectJ compiler. I personally have some doubts about annotations, if you don’t use them well they can really pollute your code.

After a short break the subject moved to the CommonJ WorkManager. The CommonJ WorkManager is very specific for Websphere and Weblogic users. I think I will keep using Quartz when it comes to scheduling instead of this WorkManager, it seemed a little complicated for simple problems.

The last part of their presentation covered JMS and Message-Driven POJO’s. This sounded very interesting because in one of our current projects where we have to deal with JMS in the near future. But it sounded more interesting than it really was, Juergen Hoeller kept talking about the differences between the two JMS specifications without really explaining the Message-Driven POJO concept.

In the afternoon there was a session about the Java In Action session by Dion Almear and Ben Galbraith. This session was very good. It was technical and very practical also. They started with a very familiar example of Ajax: form completion. Every step of setting up a form supported by Ajax was covered and, best of all, everything was coded live on stage.

After the introduction, they shifted to the most famous Ajax business case, Google Maps. The first surprise was that GoogleMaps isn’t really Ajax at all. It is just a very sophisticated usage of Javascript and HTML <div> elements. Then they proceeded to reimplement the whole Google Maps functionality, including drag-and-drop and zooming, live on stage in a few simple steps. This was all very impressive.

At the end they showed some frameworks that can make using Ajax a lot easier for developers. I personally prefer DWR. This framework lets you make use of Ajax in very (Java) natural way without having to make lots of changes in your classic web tier.

The next day, the first conference day, some of us went to the keynote by Jeff Jackson vice-president of the Sun Developer program. His talk was a little bit boring, but T-shirt launcher was nice. Every time when there was Duke on a slide a T-shirt signed by James Gosling was launched into the audience.
Always fun to see a bunch of geeks getting excited by a free T-shirt.

Later on I visited a session about JMX. In some way JMX always draws my attention. Although I never used in a real project it still stays a promising technique for managing some part of your configuration. I say promising cause I see and hear a lot of people talking about JMX, but just a few people really used it around projects.

For the last session that day I went to a session about ‘Continuous Integration’. The title itself wasn’t really exciting but it’s always fun to see someone tell story and try to convince a large audience about the advantages of using Continuous Integration during your projects. To have a continuous and quick feedback loop is so powerful and really increases the quality of your codebase. As an example that a continuous build server also can be fun, he showed usage of two lava lamps, red and green, which are connected to the build server.

If a build failed, the green lamp was turned off and the red one turned on. Even a manager can now see there’s problem. The fun thing was, before a lava lamp really gets warmed up this will take some minutes. Their sport was to fix the build before the red lamp was really on and if lava lamp is turned off before it’s warmed up, you’ll have some nice figures in the pasta in the lamp. Now they tried to keep this figures as long as possible and the only thing you can arrange this, is not to break the build. I know you must be a really big nerd to like this, but I think it’s a very nice and simple way to show that your project tries improve it’s quality. For the people who like the idea, on the site of pragmatic programmers there’s a list of components and instructions how to build your lava lamp enabled build server.

All the Javapolis presentations will become available in the next couple of months on various Java comnuity sites like TheServerSide and the JavaLobby.