I was so fortunate to be invited to present at the 10th edition of the QA&TEST. The conference focuses on QA and testing for embedded systems, an area I know little about. Though, when I first started testing in the late nineties, I was so lucky to work with a gentleman that had lost his job at Fokker (the airplane manufacturer had just gone bankrupt). At Fokker he used to test instruments used in the airplanes, like altitude meters, speedometers and other fine machinery. He knew very little about computers and was struggling with was to me basic stuff (being a rookie myself), but he was also one of the most structural and detail critical people I ever worked with. Besides loads of test techniques and responsibility for my work, what I learned from that experience is that testing is testing and the same rules apply in most fields. So after contemplating the above I happily accepted the invitation. What I experienced during the conference is that testing embedded systems is about software testing and lucky for me, in this field Agile methodologies and test automation can be applied just the same.

Automated Reliability Testing via hardware interfaces by Bryan Bakker

How test automation can successful be applied was highlighted by fellow speaker Bryan Bakker who did a great presentation of a case study on how his team. The team was requested to improve the reliability of a system and call it in an act of rebellion kept on increasing test automation rather than adding new functionality. This resulted in (if I remember the correct figures) a savior of over 1.2 million euro’s on damage that bugs would have otherwise caused in production. The result was so spectacular that they were granted extra budget to continue with their work, but also to apply the approach in other projects. Interesting takeaway from his presentation, that seems a bit more specific for embedded systems but might apply in other area’s as well, was a smart test-scheduler they had introduced. The product under test was a medical x-ray device that under circumstances would get an overheated engine, it took hours for it to cool down again which made it impossible to continue the test run. Whenever this was detected they would automatically switch to other types of tests, that were less dependent on the engine and could run until the engine had cooled down again. A simple but effective time-saver coming from a pragmatic mind.

Continuous Quality Improvement using Root Cause Analysis by Ben Linders

A session I unfortunately did not actually attend to, but that I had an interesting discussion about with the presenter, was the conference closing presentation by Ben Linders called “Continuous Quality Improvement using Root Cause Analysis”. He claims that a team can accurately predict the number of bugs it is going to make during a sprint and he has developed a method to help reduce this number using root cause analysis. I found this a fascinating and somewhat controversial idea, I have yet to meet a developer (specially the ones in Agile projects) that can admit let alone predict they’re making bugs. But as I understood it works in a similar way as predicting velocity, you get more accurate as you go, using historic data from previous sprints.

Runaway Test Automation Projects by Michael Stahl

The presentation of Michael Stahl titled “Runaway Test Automation Projects” seemed less relevant to Agile environments (at least the once I worked in), he however pointed out lots of valid risks that exist when doing test automation. Main take away for the audience in my opinion was the point that test automation should be treated as “regular” software, creating unit tests, applying quality standards and using version control. Things that in my experience sound like common sense, but never the less very true.

Pushing the Boundaries of User Experience by Julien Harty

The presentation that was probably of the most added value to me was called “Pushing the Boundaries of User Experience”, by Ebay’s Julien Harty. He had an enlightening story on automated user experience testing. With crawlers like Crawljax analysis of the dynamic (Ajax) behavior of a website can be done. With static analysis accessibility issues can be found, stuff that can be very important if you want to comply to WCAG guidelines, which hardly ever gets proper attention. Accessibility is often seen as something to facilitate a minority, but from a development point of view it helps to improve testability of the product What makes accessibility testing feasible from a business perspective is that it actually helps to improve search engine optimization, which increases the visibility of the site in search engines.
He explained how at Ebay they run automated tests for improving usability and accessibility, but also for finding layout issues and browser dependencies in a cost effective way.
Layout bugs can be found in an extremely simple but effective way using FightingLayoutBugs. Let me describe this in a little more detail.
How does this work? First you need to know which pixels belong to text, second you need to know which pixels belong to a horizontal or vertical edge, now if text pixels and edge pixels overlap you have a layout bug. Sounds pretty simple and it actually is!
How is text detected? All text of a page is set to a white font color and a snapshot is taken. All the text on a page is set to black and a snapshot is taken. Now when the two are compared, all different pixels are probably text.
How are horizontal and vertical lines detected? First set all the text on the page to transparent (btw setting the text color is done using jQuery), so now there are only graphics, take a screenshot. Now it’s identified which sequences of pixels have a certain minimal length and the same or very similar color, only those that have a high contrast to the left or to the right are selected. The same approach applies for horizontal lines. Now compare the outcome with the identified text. When text and lines overlap there’s a layout bug (which happens to be automatically reported).

During a life demo Julian ran some of these tools against the conference website. With web-accessibility-testing pointed out how bad the accessibility was for people that rely on tabbing to navigate through the screen. Static analysis of the site pointed out the page contained url’s with duplicate or no alt texts. Stuff that seems of minor severity but for disabled people essential. He concluded by revealing a security issue that caused some hilarity. He gave away the most lucrative reduction code for registration. It turned out all reduction codes for the conference were hardcoded in the Javascript in the page source.

All together it has been a valuable learning experience, beside the fact that a visit to Bilbao alone is definitely worth the trip!

Andrew’s agenda

What I saw (and, for the curious, what I missed)¹:

Wed

• Keynote: Java, the Platform for the Future
• Keynote: Java EE 6 and GlassFish V3: Evolution of a Platform
• Agile Mythbusters
• The Concurrency Revolution
• Traditional Programming Models: Stone Knives and Bearskins in the Google Age
• Keeping Your Options Open, Even if the Cloud is Not
• Distributed Programming the Google Way

Thu

• Keynote: Do we really know how to develop software?
• Keynote: Filling The Professionalism Gap
• The not so dark art of Performance Tuning
• Google Appengine Java – Groovy baby
• Traditional Programming Models: Stone Knives and Bearskins in the Google Age
• Using BTrace and DTrace to Instrument and Analyse Java Applications
• Performance for the performance-shy

That announcement

Yes, it’s somewhat ironic that I missed the big announcement of the conference: that closures may sneak in to Java 7 after all. In fact, I had decided to forego Mark’s talk in the moderately firm belief that there wasn’t going to be anything in it he hadn’t presented elsewhere or blogged about before. So much for instinct.
Needless to say, news filtered through pretty soon, though, and of course almost everybody has an opinion about this unexpected (to say the least) development.

Some samples² of what they might look like

Collections.sort(strings, #(String a, String b) a.length() - b.length());
Collections.sort(strings, #(String a, String b) { return a.length() - b.length(); });

@Shared int clickCount = 0;
button.addActionListener(#(ActionEvent e) { clickCount++; });

What’s buzzing

With Sun and Adobe “premium partners” of the event, it was hardly surprising to come across quite a number of JavaFX and Flex talks. Oracle’s entry into the fray was slightly more intriguing, focussing as they did on new features in the upcoming WebLogic releases.

Needless to say, that was slightly more enterprisey than one is used at developer conferences; I personally was intrigued to see them starting to get interested in the deployment space with an initial offering that makes it possible to create virtual machine images from configured systems.
From what I saw, there are still some important concepts missing there (which, naturally, we at XebiaLabs are trying to address with Deployit), but it’s interesting to see the middleware vendors beginning to pay attention to this.

Sponsors aside, the conference’s main threads, as far as I could see, were the “usual suspects” of concurrency, OSGi, Cloud and Scala. Of these, concurrency/distributed programming and OSGi seem to be gaining a little more enterprise traction (in terms of the number of people who actually appear to be using them for real projects); Scala and Cloud remain things to be “interested in” and “play with”, but still retain a slightly experimental flavour.

I can’t say, though, that I came across anything that hasn’t been around the conference block for at least a year. No real surprises, in other words.

Highlights

Most Professionally Thought-Provoking

Joint award here to Ivar Jacobson and Uncle Bob, both of whom presented a rather critical view of the current state of the software development industry and made some thought-provoking remarks.

The main gist of Ivar’s talk was that software engineering as a discipline lacks any kind of solid theoretical foundation, lurching from (methodological) fad to fad rather than developing a cumulative body of best practice knowledge.

His approach to avoiding what he considers as a recurring tendency to throw out the baby out with the bathwater is to break methodologies such as Agile down into sets of practices. In this framework, these constitute the “atomic” entities, and supposedly make it easier to identify common features of different methodologies and incrementally improve them.

Whether the Iterative practice in e.g. Scrum is really the same as Iterative in a different methodology, or whether processes are really just “bags” of practices to which you can add others without disturbing some inherent balance, are questions left unanswered.

In any case, Ivar has drummed up a bunch of Thought Leaders and is issuing the following call to action, to which all developers are apparently invited to contribute.

By contrast, Uncle Bob’s talk focussed less on the process and more on the technical side of software engineering. His central tenet: developers need to learn to behave more like skilled professionals. Which to him means, amongst other things:

• Learning to say no (how often have you heard a doctor say: “Fff, it’ll be tough, but we’ll try and get your liver done by tomorrow?”)
• Demanding a right to a technical career path that suitably recognizes
  accumulated skill and experience (rather than forcing developers to become managers to progress)
• Always aiming for a “professionally ethical” level of code quality (independent of business value)

It intrigued me to note that the attitudes that Bob was referring to are most commonly found in guild-like professions (such as law or medicine), whose practices are usually based more on experience and tradition than hard science. But let’s keep the “software development: craft or science?” debate for some convivial pub session, shall we?

Most Technically Thought-Provoking

Looking back on it as a whole, Cameron Purdy’s talk was no more or less than a well-delivered, knowledgable presentation on the benefits of event-driven architectures. The observation he used as an introduction, however, I found fascinating: simply learning to program is bad for scalability.

The argument goes a little bit like this: in almost all the languages you might learn as a beginning programmer, you can write something like:

  int total = 0;
  int i = 10;
  addToTotal(i);
  // now total = 10
  i = 20;
  addToTotal(i);
  // now total = 30

Trival. But hidden amongst our “natural” interpretation of what code will do are the following expectations:

• the statements will be executed
• the statements will be executed in the order written
• the statements will be executed exactly once

Cameron’s point is that, if you consider distributed systems, these requirements – guaranteed execution, global ordering and exactly once processing – are just about the most stringent, and hence costly, expectations one can have.

Systems which are fault-tolerant, free of race conditions and idempotent have a much better chance of scaling and distributing well, and one of these reasons this is hard is because it goes so deeply against the grain of the intuitive picture of how programs run that are ingrained right from the first line of code we write.

Best Technical Content

Trust Google to put together the talk with the most varied range of “cool stuff”. Gregor Hohpe gave a very engaging presentation on MapReduce, Bigtable, Sawzall and others, interspersed with mentions of things like Bloom filters and other interesting theoretical concepts.

The common theme here was definitely trading off functionality for scalability, with the conclusion that we can often get away with much less functionality than one might expect. Sawzall, for instance, does lightning-fast distributed aggregations, but, being history-less (you only see the current row), can’t even do duplicate checks.

So what do you do if you discover that the engineers over at the Mountain View chocolate factory have sacrificed even the tiny little bit of functionality you need? Can you hand back a bit of that Bigtable scalability for multi-row locks, or are you straight back to Oracle 11g?

Well, it appears that Google does have a transactional engine on top of Bigtable, after all. But that’s certainly not open source!

Best Presentation

The prize for sheer viewing pleasure goes to Doug Tidwell’s talk on SimpleCloud. Humourous, slickly presented, with a good mix of examples and concepts served in manageable chunks, I certainly enjoyed this one. Including a recurring joke (in this case, Adam Koford’s set of HTTP response code illustrations isn’t a new idea, of course, but this one was actually topical and funny.

In terms of the content, SimpleCloud looks like a pretty decent cloud API. Quite how useful a PHP library can be for Java developers remains to be seen; if you’re looking for a powerful, concurrent Java cloud library with support for lots of cloud providers try jclouds³.

Lowlights

Most Annoying Trait

To presenters from North America, a polite request: if at all possible, please remove jokes or other references to intrinsically North American popular culture.
Having to sit through a five-minute explanation of who a particular character in an American sitcom in the 70s was, or what the point of a certain gameshow was, isn’t a great start for a presentation that was supposed to begin with a quick joke and then get on to the topic the audience is actually interested in.
Thank you.

Biggest Disappointment

Working for a company that is widely known for its Agile expertise, I was looking forward to Scott Ambler’s promise of investigating some of the myths and rumours surrounding Agile to the scrutiny of cold, hard statistics.
What we got, however, was a lightning dash through the self-reported results of a number of surveys, without so much as a hint of analysis or further explanation.

Whilst it’s certainly not uninteresting to learn that more than 50% of Agile teams do in fact do some up-front design, what I would very much like to know is whether up-front design actually leads to more successful projects! Suffice it to say that my colleague and I left the hall with many questions and few, if any, answers, and that’s not taking into account the confusing presentation of many of the numbers.

A note on Performance Talks

As you might have guessed from my conference agenda, I’m quite interested in all this concurrency and performance stuff (it probably wouldn’t surprise you either to learn that I’m also trying to help with Multiverse, a Java STM implementation). But after this round I think I’ll be skipping performance tuning presentations for a while.

Not that Kirk’s or Holly’s presentations were bad or even boring, they were simply going over the same basic “performance tuning 101″ stuff that is talked about every year, at every conference.

Now, these sessions are usually packed, so there certainly still seems to be plenty of demand for learning how to get a thread or heap dump or how to write a JMeter test plan.

Naturally, I understand that, because performance tuning is so specific to the application at hand, it’s difficult to give more than general information and guidance. But at the end of one of these sessions I always feel I’ve hardly scratched the surface of these experts’ knowledge of application performance.

What I would love to attend, for instance, is session during which I could simply watch these performance experts at work, attacking the problem of tuning a real application “from scratch”.
The point would not be to see to what extent the application could be speeded up – in fact, that would be completely irrelevant. But I think there would be a lot to learn from seeing how a performance tuning professional goes about such a task in practice: where to start measuring, what tools to use, how initial hypotheses are formed etc.

But of course it’s also not realistic to expect people to give away the “trade secrets” that allow them to earn a living just like that.

Oh, and to all the JVM vendors out there: please consider coming up with some standardized formats for profiling information! It’s such a pity to watch demonstrations of really useful-looking tools (such as IBM’s Health Center) only to discover they can’t work with the data produced by the JVM your client is using.

Quote of the conference

The rumour that Mark had come up with the new closure proposal syntax on the plane over to Belgium certainly caused a few chuckles. My favourite, though, was a remark attributed to Larry Ellison, who in response to possible demands from the EU to sell off mySQL is supposed to have said: “The ‘my’ in ‘mySQL’ means Larry’s SQL.”.

A final note

Hats off again to the Devoxx team. Content seems to be more or less on a par with JavaOne, and yet the attendance fee is still roughly €1000 less than almost all other conferences.

It’s not perfect, of course: the wireless connection is just as continuously unavailable as elsewhere, and the food-loving Belgians are unlikely to consider the catering the pinnacle of Belgian culinary tradition. But after all, for €1000 you could eat and drink yourself pretty silly in Antwerp’s many good restaurants and bars, and still have enough left over for some lightning-fast internet.

For me, still the European conference to beat.

Last week I attended JAX ’09, the Java User Conference in Mainz, Germany.
Or rather “conferences”, because once you’re there JAX is indistinguishable from something called SOACON and the Eclipse Forum Europe, which officially take place in parallel.

Vibe

Compared to last year’s Devoxx (a.k.a. JavaPolis) and certainly events like FOSDEM, the mood at JAX seemed rather subdued. Since I hadn’t been to JAX before, though, it’s hard to say if that was the recession biting or just the norm for this conference.
I overheared one of the organisers mention that attendance was down on previous years, though. Certainly, for a conference of this size – 08:30 to 22:15 daily, with up to 12 concurrent sessions – it felt a bit empty. The “expo” area looked especially quiet.

Apart from some of the “usual suspects” (Neal Ford, Ted Neward, Peter Kriens; SpringSource, JBoss, dynaTrace), the vast majority of the audience and exhibitors were German developers and consultancies, respectively, giving the conference a strong “national” feel.

Hot

Buzzwords at this year’s JAX (coinciding, unsurprisingly, with the conference’s “Theme Days”):

• Agile
• Scala
• OSGi
• DSLs

Slightly disappointingly, these were all things that have been around for a while.

Not

• Web application development in Java
• ESBs
• Cloud

Keynotes

Ancient Philosophers and Blowhard Jamborees

Neal Ford’s Ancient Philosophers and Blowhard Jamborees keynote was an entertaining, as ever, look at software development’s past, present and future.

Neal laments our continuted inability to learn from the past – our blissful ignorance of software development’s “lore“, as he put it. As a result, we not only keep reinventing the wheel; instead of tackling the actual core problem at hand, we waste a lot of time and energy grappling with enormous white elephants architectural patterns, whose complexity is essentially peripheral to the issue to be addressed.
Whenever we’re struggling with SOA, ESB or one of the other Rube Goldberg devices enterprise middleware solutions, we’re paying, in his words, “complexitax”. And our ignorance is keeping the tax rate high.

Perhaps to underline the message that we urgently need to become more aware, critical and, ultimately, better programmers, Neal pointed out that, if we don’t, Chindia (ThoughtWorks-speak for the China/India juggernaut) will. They definitely have the capacity: as Neal mentioned, India’s top 1% = 11,298,661 people.

That certainly sobered everyone up a bit.

Fette Maschinen brauchen schlanke Software

Fette Maschinen brauchen schlanke Software (“Heavyweight machines need lean software”) was Klaus Alfert’s take on the impact of multi- and massively multi-core architectures on our future programming styles and requisite skills.
Using the well-known graph of Amdahl’s Law to demonstrate that additional processing units won’t give us extra performance just like that, he posited that Von Neumann architecture itself, with it’s mutable memory cells, is the root cause of many of our concurrency struggles.
Referring to John Backus’ famous Turing Award Lecture, he singled out three approaches as key in the quest for safer, simpler concurrent programming:

• Functional Programming
• Actors (cf. Erlang)
• Software Transactional Memory¹

Spotlight: Agile Documentation

Uwe Friedrichsen gave a interesting talk on how to approach documentation requirements in an Agile project. Some key points:

• Production & maintenance costs consume 85% of a project’s budget. Compare this with Agile’s focus mainly on the development phase.
• A complete product is more than just software; maintenance often relies heavily on documentation, for instance. But many “standard” documentation templates comprise 1000-1500 pages – that can’t all be valuable.
• The purpose of documentation ultimately is answering users’ concerns, so good “stakeholder engineering” is critical.
• Given changing requirements and functionality, it doesn’t pay to start producing documentation too early – do ensure negotiate structure and content are agreed upon, though.

Slides are only available in German, unfortunately.

Spotlight: OSGi New & Noteworthy

Coming up in OSGi 4.2, amongst others

• Service Hooks
• (updated) Declarative Services

In the pipeline are also

• Distributed OSGi
• Transactions
• JPA
• more J2EEy stuff

“A Component Model for OSGi” (a.k.a. RFC 124 a.k.a. Spring DM) doesn’t appear to have made it², even though it was mentioned in the last early draft. For a quick summary of the original early draft, have a look at Mirko Jahn’s blog post.

With support for Enterprise Java development coming more and more to the fore in OSGi, Kai Tödter and Heiko Seeberger’s comparison of the current DI offerings was timely. Martin Lippert’s slides (from another talk) contain some further examples.

The reference implementation of Distributed OSGi is being provided by CXF.

Gripes

Sprechen Sie Deutsch?

By far the most surprising realisation came when I walked into the first talk and discovered that it was in German. Fortunately for me, that’s my mother tongue, but international visitors will have had a somewhat ruder awakening.

This was all the more surprising given that I don’t recall seeing an explicit mention of this anywhere on the conference website, during the registration process or even on arrival!

OK, the summaries of many presentations were in German, but on the German version of the site that was hardly surprising. One might say that the fact that the abstracts were in German even on the English pages should have served as a hint, but 9 out of 10 times that simply means that they simply didn’t get round to translating it, in my experience.

To be fair, the Eclipse Forum Europe talks were all supposed to be in English (hardly a surprise for a conference with Europe in its title, I’d say).

However, at least one of the presenters, when asked by a Greek former colleague of mine to please switch to English, responded that he’d only heard about that requirement late at night on the day before his talk, and thus wasn’t prepared. Hm.

To cut a long story short, I would simply expect all presentations and materials at an international conference to be in English, and I found JAX highly unprofessional in that respect. I’d be curious to hear what any of organizers would have to say about this.

Working Offline

Pleasing as it is to see nations refusing to live up to tired stereotypes, some aspects of the organisation could have done with a leetle more “German efficiency”.

Before I am accussed of nitpicking, is it really that unexpected that attendees at a software development conference might be bringing portable devices, that may require power sockets to be recharged? Or that, in today’s networked world, these attendees may wish to connect to the internet?

As it was, most venues had barely a couple of coveted rows with extension cords, and in some there was no power because the fuses blew (to be fair, this was quickly addressed as the week progressed).
With respect to connectivity, the Wifi connection, on the rare occassions when it was up, was so slow it was barely distinguishable from being offline. So much so that most presenters crawled through the online portions of their presentations via GPRS, rather than risking a wireless connection.

Not big things, really, but they stick in one’s mind.

PS: To all the Dutch readers, have a great Queen’s Day!