The Conference

If you’ve never heard of QCon before, the conference bills itself as a conference for and by software developers and architects of any “denomination”. Walking the hallways you are equally likely to encounter a Rubyist, Javaan or .NET afficionado. This refreshing diversity leads to interesting presentations and coffee-corner conversations. For me as a Java developer, it also meant that the news about Java 7′s closures by and large passed me by.

Given it’s target audience of predominantly teamleads and architects, QCon has less technical depth than, for instance, the Devoxx conference. But with entire tracks dedicated to Ruby and emerging languages (including some C#/F# sessions), it certainly has more breadth. For me personally this is one of the key reasons to attend.

Keynotes

The keynotes this year were a bit of a mixed bag. On Wednesday, two venture capitalists spoke about what they looked for in startups when deciding where to invest. The process they described resembled the Lean Startup ideas of iterating as fast as possible with direct customer involvement. Interestingly, they mentioned Scala and Groovy as developments that they as VCs keep a close eye on.

Don Box experimented with Twitter during his keynote. Not saying a word, he wrote tweets on his laptop for the first 20 minutes. An interesting experiment, but it didn’t manage to hold the attendees’ attention. When the subsequent code demos either failed or proved unable to bring the audience back, Don himself also ran out of steam and brought the keynote to an abrupt and disappointing close. Still good of him to try something new.

The last keynote of the conference was by David Chappel of Oracle who presented and then demonstrated Oracle’s cloud tooling. Way too Oracle-centric for my money.

For next year, I really hope QCon will do away with the vendor keynotes, even if this means going back to 2 keynotes like the last few years. Someone like Kent Beck or Uncle Bob has much more interesting things to say.

Highlights

Dan North’s presentation about a challenging agile project was great. It was filled with lessons about transitional architecture and agile adoption and plenty of humor to boot.

In Lessons learned from Architecture Reviews Rebecca Wirfs-Brock discussed not only how to perform an architecture review, but also how to effectively present an architecture depending on your audience and the political environment.

Eric Evans talked about incorporating modeling and design into the agile development process and presented an early draft of his model exploration process. Eric uses this process when doing DDD consulting for clients and is useful for anyone applying the technique.

There was a lot of interest in lean software development and kanban as well. I attended presentations by Jeff Patton and David Laribee which provided useful practical tips and real-world experience. Though I didn’t attend it myself, I heard a lot of good things about Henrik Kniberg’s talk as well.

One of the most impressive tracks at QCon is Architectures you’ve always wondered about. I’m always inspired by seeing how sites such as eBay or LinkedIn handle their scaling requirements. This year’s session about Amazon S3 was equally impressive, describing design patterns for building applications that can guarantee uptime in the face of hard drive, network or machine failures. To test their service, Amazon regularly holds unannounced “game days” where they take an entire datacenter offline to see if their architecture can handle it. That is some serious testing.

Trends

During the conference, I noticed a lot of interest in:

• teaching people the craft of agile software development. Presentations discussing personal traits required for programming and software apprenticeship were the result.
• JVM-based languages like Scala and Clojure. This is hardly news, but what surprised me was the level of interest there was from non-Java developers. Several people I talked to told me that they were experimenting with these languages in their spare time while working .NET at their dayjob.
• Continuous deployment (meaning continuous deployment to production).

Bucking the trend I’ve been hearing about at other conferences, the wifi at QCon actually worked without a hitch for me.

Conclusion

For me, QCon was about re-discovering the agile software development craft this year. Talking with practitioners and hearing their experiences gave me lots of new ideas and inspiration to bring to my own projects. That said, most of the tracks and topics feel like they’ve been around for a while. Next year I’m hoping to hear about the “next big thing”.

In my current project, the system we are building has been accepted by the client and is in production. At the same time, however, new development on the software is taking place. The bugs and user stories resulting from the first and second activity end up on the same product backlog and are worked on by the same Scrum team. For the user stories, we include coding, development testing and documentation in the Definition of Done. This has worked well for us and allowed us to create the system in the first place.

However, the bugs are a different story. These are defects in the already existing software that were found by either an external testing team or in production. At first, we were using the same Definition of Done for the bugs as for the user stories. When delivering software fixes for these bugs, our customer would regularly ask us a series of
questions we had no answers for:

• which versions/applications are impacted by this bug?
• which versions need to be patched?
• does this affect the interface between applications A and B?
• is there a workaround for it? Has it been documented?

We realized the Definition of Done for the bugs needs to be different from the ones used for regular user stories. By including the questions above, we create transparency about what we need to consider when solving bugs and are able to better meet the customer’s expectations.

What do you think? Is it a good idea to use different DoDs for user stories in one backlog?

Team members on a software development team are dependent on each other to achieve their goals. This interdependence forces team members to work together. Of course this is easier said than done. One of the things that makes this hard is the way people respond to problems. Typically, they go through the following steps:

1. Lay blame (“Which one of you code monkeys broke the build?!”)
2. Justify (“It’s always broken anyway…”)
3. Shame (“I must be the worst developer EVER!”)
4. Obligation (“I guess I have to fix the build since that’s what we agreed on.”)
5. Take responsibility

Only the fifth step is a good attitude for learning and growth, but in order to arrive there, the first steps will have to be passed. Being aware of these steps can help you take responsibility more quickly.

Christopher showed data about successful teams and the factors that can predict their success. The interesting thing is that the top 3 factors were trust, goodwill/cooperation, and a shared purpose. Technical skills appear somewhere around the 15th spot on the list. This surprised me since in my experience, technical expertise plays a larger role in project success or failure than this spot leads me to believe.

One thing that is often forgotten is that the “project team” really isn’t a team at all yet — they are a group of people who have been put together to accomplish a goal. Chances are that at least some of the people have not worked together before and that there is no common ground for them to become a team. According to Bruce Tuckman, their evolution as a team can be divided into four stages:

1. Forming
2. Storming
3. Norming
4. Performing

Forming

When a team first gets together, it enters the forming stage. Team members are usually on their best behavior, working mostly independently. The team members get to know each other and learn about the project goals and issues.

Storming

In this stage the team members open up to each other and confront each other’s ideas and perspectives.

In some cases this phase can be resolved quickly, depending on the maturity of some team members. Immature team members will begin acting out to demonstrate how much they know and convince others that their ideas are correct. This phase is necessary to the growth of the team, but can be unpleasant to members of the team who are averse to conflict.

Norming

When a team enters the norming state, team members adjust their behavior to each other and agree on how to work together.

Performing

Some teams will reach the performing stage at which point the team is able to function as a unit and finds ways to get the job done smoothly and effectively without inappropriate conflict or the need for external supervision. Team members have become interdependent.

Team norming and chartering

To assist in the norming stage, it helps to conduct a facilitated norming session with the entire team. In this meeting, the team performs two activities. The first, team norming, serves to establish working agreements that enable the team to work together. Rules, values and professional behavior fall under this heading. Team chartering, the second activity, deals with shared methods and tools. The resulting agreements should be prominently posted (on the wall or a Wiki) and be visible to everyone. These agreements provide the basis for effective team work and can be referred to in the case of disagreements.

As an example, here are some agreements the team came up with when we held a team norming session at one of our project:s:

Norming example

Core Hours

• Everybody should be present from 9h30 – 16h00

Technical Discussions

• Important discussions should be in core hours, to prevent absentees.
• Results from the discussions should be on the wiki.
• White boards sesssions should be documented on the wiki.
• The format used for this should be: Problem, Rational, Solution, Other discarded options.
• Absentees should:
• Trust the rest of the team.
• Reopen the discussion (after reading all documentation) if they strongly believe a mistake has been made, with a representative of the group that had the original discussion.
• Whiteboard discussions should be on skype (with the offshore team) when possible.

Cell phones

• Cell phones are on during working hours
• Personal calls are taken outside of the team area

Chartering example

Code Quality

• Everybody should check the code quality when they check in code (use CCs maven reports).
• Everybody can suggest changes to quality standards.
• We strive to prevent technical debt.
• Discuss refactorings with as many people as appropriate (could be just pair).
• Try to include refactorings in the User Story.
• Balance refactoring with adding new functionality.
• Javadoc: We follow the company checkstyle standard.
• We have a standard header, which contains a copyright notice.

Quality targets

• Test coverage: 90% overall. UI code should be tested by other means (logic should be unittested).
• We strive to get the number of checkstyle violations as low as possible.

Continuous Integration

• We try to check in at least once every day
• If the build breaks, we fix it as soon as possible

Conclusion

In my experience, team norming and chartering helps to give a newly-formed team some common ground from which to start. Open discussion of the practices and processes used to develop software ensure that everyone has a chance to contribute. The results from such a session can be referred to in case of disagreements and when new people join the team.

Two presentations by eBay architects gave some insight into what must surely be the most scalable Java application on the internet today. eBay architect Dan Pritchett described how their architecture deals with latency. Instead of using ACID for their application, ensuring atomic and consistent data at all times across all databases, they employ the BASE (Basically Available, Soft-state, Eventually consistent) paradigm to improve availability and response time by sacrificing consistency and atomicity.

Randy Shoup discussed eBay’s architecture in more detail. His presentation, The eBay Architecture: Striking a balance between site stability, feature velocity, performance and cost, highlighted the strategies eBay uses to scale their application. By applying functional segmentation (dividing the application by functionality) and horizontal split (for each piece of functionality, use multiple instances that are interchangeable) their architecture is able to scale to the required number of users. When you’re dealing with sites this size (to give you an idea, eBay has 16,000 application servers and produces 1.5TB log data per day) everything is custom — deploying new versions of the application, analyzing log files, persistence of data using partitioning, eBay uses custom built tools for everything. Very interesting to see, even if we don’t all build applications of this size.

DSLs
While not a separate track, DSL (Domain-Specific Languages) were an oft discussed topic. One presentation on this topic, Building DSLs in Static and Dynamic Languages presented an overview of the motivations for using DSLs and categorized them as either internal (meaning implemented in an existing language) or external (using a custom-built lexer/parser). The former often takes the form of a fluent interface — a human-friendly, readable API. The presentation’s Java examples showed use of method-chaining to make statements more readable. A testament to the attention for this topic was Joshua Bloch’s statement that he was preparing a JSR that aims to, among other things, improve support for method chaining in the Java language by implicitly returning this from void methods.

Non-Java
QCon provided a broad view of enterprise IT, not focusing on one language or platform. Instead, tracks on advances in both .NET and Ruby were presented and well attended. To underline this, participants in a panel discussion on the future of Java (which included Joshua Bloch and Charles Nutter) agreed that the Java language itself will become less important than the Java platform. Languages like JRuby and Jython are maturing and gaining in popularity. Rather than languages, tools and frameworks will determine which computing environment will “win” in the long run.

Agile
Of course, the agile movement was not missing from this conference. A track about the current “Challenges in Agile” brought a number of agile-focused presentations. The most interesting of these, Agile software development in the large discussed issues surrounding running agile projects with multiple teams, often geographically dispersed. Since I’m involved in just such a project, the presentation yielded little new insights, but confirmed many of our practices.

Conclusion
The technical content was well-presented and, overall, quite in-depth and useful. Of all tracks, the Architectures you’ve always wondered about track was most interesting as well as most popular. The track on Architecture Quality did not focus on software architecture quality as I’d hoped, but instead discussed qualities of system architectures.

An improvement would be to include sessions on topics like team dynamics or estimating and planning, which are certainly useful for team leads and project managers.

All in all, this was a good conference and more than enough reason to look forward to the next QCon, next year in London.

We use the following process for this. At the end of a sprint, all unresolved bugs classified by the testers as major or higher are entered onto the product backlog as separate items. Issues of minor importance are collected in our bugtracking tool. At the planning meeting for the next sprint, the product owners select the highest priority items (including bugs) from the product backlog for inclusion on the sprint backlog. Items that are not selected remain on the product backlog, possibly for next sprints.

This process provides transparency about the workload left to the product owners, both in terms of user stories to be developed as well as outstanding issues. They can decide on the relative priority of these, giving them full control over the sprint scope. It also creates a bridge between our issue tracking system and the Scrum process, ensuring that these are not two separate worlds.

For this to work it is critical that the testers and the product owners agree on what constitutes a major versus a minor bug. If this is not the case, either unimportant issues show up on the product backlog (and will most likely remain there) or, worse, important issues are left in the minor category, with no visibility to the product owners. A regular review of minor issues with the product owners mitigates this risk and provides further transparency to stakeholders.

The contest runs for a whole day, with each team competing in 5 tasks (out of 6, each team was assigned one task they would not be competing in). Each task contains one or more Java source files, with one class or method still to be implemented. The client software (an applet) behaves as an IDE (though no handy Javadoc lookup or code completion) and is used to compile the code, run the testcases and submit the results.

Our team (named Inspiring 42 after a counting mishap ) participated for the first time. Although we did have experience in other programming contests, the MoJ format was new to us: the realtime program/compile/test cycle, the short timeline for each task (30 minutes) and the large visible clock (and accompanying sound effects, like the crash of a gong at the start of the round and an annoying bleep-bleep sound when another team submits a solution).

The organizers published a preview of the cases on the MoJ site days before the event. That a lot of the contestants had looked at these and guessed what the tasks might be showed during the first assignment (validating Sudoku puzzles). Solutions for the Sudoku problem were entered very quickly. We had also done our homework and managed to be the first competing team to succesfully submit a solution.

For the second task we had to implement a simple load-balancing algorithm. Again, we finished first. Everything was looking good and the main price (a SUN Opteron workstation) was almost ours!

The third task had to do with regular expressions and this was really tough for us. At the end of the alotted 30 minutes we still had not found the correct regular expressions to solve this problem. Funny enough, neither had the winning team — they solved the problem by creating an anonymous inner class that parsed the input using a StringTokenizer. Hardly what the organizers had in mind, but it worked and earned them 20 points. I just wish we had thought of it!

Obviously, this debacle caused us to drop to the number two spot. Worse, the team in the #3 position did not compete in the first task, so they could pass us if they performed well in the task we would not compete in. This was the fifth task: performing MD5 hash calculations to find suspect employees. We did well (finishing first again) but that did not score us any points since we weren’t competing. So now we were in the #3 position.

The last official assignment, AaiRobot, was a funny one. The idea was to guide a robot through a maze to it’s home location. There was very little information to go on, though, only a bump sensor, light sensor (to sense the color of the tile the robot was on) and a test to see whether or not this was the destination square. There was no indication of how far the robot was from the final square.

Also, the last test case required the robot to walk along a corridor and, halfway through, turn left without any indication. After attempting to come up with a generic algorithm, we realized a different approach was needed and, using trial and error, we determined the exact location at which to turn left (using the total distance travelled by the robot) that worked for the test case and submitted the result.

Even though we gave it our best shot we never recovered from the regexp task. The robot-task was a weird one and I’m not sure I like it. In the end, we did not submit a correct algorithm but a program that manages to complete the testcases. A small change such as moving the destination square in the final test case left by one square will break our “solution”.

All in all, the contest was a great experience and a lot of fun. We’ll be sure to participate again next year. And you better believe we will have studied up on regular expressions by then!

Team Inspiring 42 a.k.a. Erik Rozendaal / Martin van Vliet