For the first problem, Google has intercepted some alien communications but the signal has weakened so much that some of the words are unclear. They now need our help to devise a clever algorithm that can tell them how many different words the signal can represent. For this they constructed the full dictionary of the alien language. As a sample they provided us with the following alien language which consists of 5 3-letter words, and 4 received signals.

3 5 4
abc
bca
dac
dbc
cba
(ab)(bc)(ca)
abc
(abc)(abc)(abc)
(zyx)bc


For my solution I decided to read the dictionary of all the words, and subsequently convert each of the signals to a regular expression by doing

val pattern = "(ab)(bc)(ca)".replaceAll("\\(", "[").replaceAll("\\)", "]").r

This lead to the following code:

object AlienLanguage {

	def main(args: Array[String]) = {
		if (args.length == 0) {
			throw new IllegalArgumentException("Could not find input file in arguments")
		}

		val input = args(0)
		val outputFile = new File((input substring(0, input lastIndexOf("."))) + ".out")
		val reader = Source fromFile(input) getLines
		val ar = reader nextIntArray
		val (tokens, nrWords, nrProblems) = (ar(0), ar(1), ar(2))
		val results = new Array[String](nrProblems)

		val words = (1 to nrWords).map(x => reader.trimmedLine).toList
		for (val i <- 1 to nrProblems) {
			results(i - 1) = "Case #" + i + ": " + solveProblem(reader, words)
		}
		outputFile write results
	}

	def solveProblem(reader : Iterator[String], words : List[String]) : String = {
		val pattern = reader.trimmedLine
		val regex = pattern.replaceAll("\\(", "[").replaceAll("\\)", "]").r

		matchWords(regex, words.head, words.tail).toString
	}

	def matchWords(regex : scala.util.matching.Regex, word: String, words : List[String]) : Int = {
		words.isEmpty match {
		  	case true => matchWord(regex, word)
		  	case false => matchWord(regex, word) + matchWords(regex, words.head, words.tail)
		}
	}

	def matchWord(regex : scala.util.matching.Regex, word: String) : Int = {
		regex.findFirstIn(word) match {
		  case Some(x) => 1
		  case None => 0
		}
	}
}

The code uses two helper classes which use implicits to add methods to a File and to an Iterator[String]. These can be found in my GitHub repository (see bottom). This code was able to solve the large input set (max. 5000 15-letter words and 500 signals) within 2,5 seconds.

For the second problem we were given an elevation map for which we had to calculate drainage basins, based on a a set of rules:
* From each cell water flows to at most one neighbour cell
* Water always flows to the lowest neighbour, in case of a tie, water prefers North, West, East, South in this order
* If no neighbour has a lower altitude, water stays in this cell, we call this a sink
* drainage systems are identified by a single unique lowercase letter, such that when all rows are concatenated, the string is lexographically smallest.

A sample input with 1 5×5 map
1
5 5
1 2 3 4 5
2 9 3 9 6
3 3 0 8 7
4 9 8 9 8
5 6 7 8 9


My solution consists of first building the map and then converting that to a “flow map”, in which every cell points to the index of the cell to which the cell drains. This flow map can then easily be transformed to a “basin map” by starting in the top-left corner and going from left-to-right and top-to-bottom through the map, assigning a unique letter to the drainage system. The code is:

object Watersheds extends CodeJam {

	def solveProblem(reader : Iterator[String]) : String = {
		val ar = reader.nextIntArray
		val (h, w) = (ar(0), ar(1))

		val theMap = (1 to h).map(x => reader.nextIntArray).toArray

		val shed = new Shed(h, w, theMap)
		val basins : Array[String] = shed.solve
		printRecursive(basins, w)
	}

	def printRecursive(basins : Array[String], length : Int) : String = {
		basins.size match {
		  	case 0 => ""
		  	case _ => "\n" + basins.take(length).reduceLeft(_+" "+_) + printRecursive(basins.drop(length), length)
		}
	}
}

class Shed(val h : Int, val w : Int, val theMap : Array[Array[Int]]) {
	val flowMap = (for (hPos <- 0 until h; wPos <- 0 until w) yield determineFlowTo(hPos, wPos)).toArray
	val basins : Array[Int] = Array.make(h * w, -1)

	def determineFlowTo(hPos : Int, wPos : Int) : Int = {
		val north = determineAltitude(hPos - 1, wPos)
		val west = determineAltitude(hPos, wPos - 1)
		val east = determineAltitude(hPos, wPos + 1)
		val south = determineAltitude(hPos + 1, wPos)
		val me = determineAltitude(hPos, wPos)
		val minAltitude = List(me, north, west, east, south).sort(_<_).head

		if (me > north && north == minAltitude) {
			convertToIndex(hPos - 1, wPos)
		} else if (me > west && west == minAltitude) {
			convertToIndex(hPos, wPos - 1)
		} else if (me > east && east == minAltitude) {
			convertToIndex(hPos, wPos + 1)
		} else if (me > south && south == minAltitude) {
			convertToIndex(hPos + 1, wPos)
		} else {
			convertToIndex(hPos, wPos)
		}
	}

 	def determineAltitude(hPos : Int, wPos : Int) : Int = {
		if (hPos == h || hPos < 0) {
			10001 // more than max altitude in large map because out of range
		} else if (wPos == w || wPos < 0) {
			10001 // more than max altitude in large map because out of range
		} else {
			theMap(hPos)(wPos)
		}
	}

	def convertToIndex(hPos : Int, wPos : Int) = {
		hPos * w + wPos
	}

	/*
     * Solve methods
	 */
	def solve() : Array[String] = {
		val basinString = "abcdefghijklmnopqrstuvwxyz".split("")
		var currentHighestBasin = -1
		(0 until h * w).foreach(i => if (basins(i) == -1) currentHighestBasin = fillFlow(i, currentHighestBasin))
		basins.map(i => basinString(i + 1)).toArray
	}

  	def fillFlow(index : Int, currentHighestBasin : Int) : Int = {
		val flowTo = flowMap(index)
		if (basins(flowTo) != -1) {
			basins(index) = basins(flowTo)
			currentHighestBasin
		} else {
			val basinValue = findBasinValue(index)
			if (basinValue != -1) {
				fillTheFlow(index, basinValue)
				currentHighestBasin
			} else {
				fillTheFlow(index, currentHighestBasin + 1)
				currentHighestBasin + 1
			}
		}
	}

	def findBasinValue(index : Int) : Int = {
		flowMap(index) match {
		  	case `index` => basins(index)
		  	case _ => findBasinValue(flowMap(index))
		}
	}

	def fillTheFlow(index : Int, basin : Int) {
		basins(index) = basin
		val flowTo = flowMap(index)
		if (flowTo != index && basins(flowTo) == -1) fillTheFlow(flowTo, basin)
	}
}

Note that the CodeJam trait which is extended provides a main method, much alike the one from the first problem. The output that is generated by the code looks for the sample input given above as
Case #1:
a a a a a
a a b b a
a b b b a
a b b b a
a a a a a

The program was able to solve both the large and small datasets in under 1 second.

The last problem was (at least for me) a hard nut to crack, though the small input was easy to solve, my program didn’t finish the large input within the scheduled 8 minutes. The problem statement was as follows:
Given a text string, you are to determine how many times the string “welcome to code jam” appears as a sub-sequence of that string. In other words, find a sequence s of increasing indices into the input string such that the concatenation of input[s[0]], input[s[1]], …, input[s[18]] is the string “welcome to code jam”.

My approach consisted of the following steps:

• first filter out all characters from the string that do not belong to the “welcome to code jam” message
• do a simple kind of run-length-encoding trick to decrease the length of the string even further (while maintaining the information)
• loop over the string and calculate the number of occurrences

object WelcomeToCodeJam extends CodeJam {
	def solveProblem(reader : Iterator[String]) : String = {
		val expr = "welcome to code jam"
		val line = reader.trimmedLine

		// first filter all characters not occurring in pattern
		val filteredLine = makeHisto(line.toList.filter(x => expr.contains(x)))
		val exprList = expr.toList
		(solve(filteredLine, exprList) % 10000) formatted ("%04d")
	}

	def makeHisto(line : List[Char]) : List[(Char, Long)] = {
		if (line.isEmpty) {
			List[(Char, Long)]()
		} else {
			val h = histoLetter(line.tail, line.head)
			val newLine = line.drop(h._2.toInt)
			newLine.isEmpty match {
			  case true => List(h)
			  case false => List(h) ++ makeHisto(newLine)
			}
		}
	}

	def histoLetter(line : List[Char], c : Char) : (Char, Long) = {
		(c, (line takeWhile (_ == c)).size + 1)
	}

	def solve(line : List[(Char, Long)], expr : List[Char]) : Long = {
		var count : Long = 0L
		if (!expr.isEmpty) {
			var restLine = findNext(line, expr)
			while (!restLine.isEmpty) {
				count += (restLine.head._2 * solve(restLine.tail, expr.tail))
				restLine = findNext(restLine.tail, expr)
			}
		} else {
		  count = 1
		}
		count
	}

	def findNext(line : List[(Char, Long)], expr : List[Char]) = {
		line.dropWhile(_._1 != expr.head)
	}
}

Now, given the fact that some people actually managed to do this in a very small amount of time, my guess is that I completely missed something which is why it couldn’t finish the large input set (lines of 500 characters). The small input set was correctly generated however. I’ve been busy this last day refactoring the third program to improve its speed, the latest version is the one shown above, who can tell me what I missed, or how it can be done faster?

For the code of the 2009 edition, and some of the problems of the 2008 edition, you can also visit my GitHub repository at: http://github.com/hierynomus. Who else competed, or who has a better solution in Scala?



]]> http://blog.xebia.com/2009/09/04/google-code-jam-2009-qualification/feed/ 5 http://blog.xebia.com/2009/07/02/shuntingyard-algorithm-in-scala/ http://blog.xebia.com/2009/07/02/shuntingyard-algorithm-in-scala/#comments Thu, 02 Jul 2009 10:21:16 +0000 Jeroen van Erp
Warning: Invalid argument supplied for foreach() in /home/blog.xebia.com/www/wp-content/plugins/autometa/autometa.php on line 303
http://blog.xebia.com/?p=2331 Last week I came across an interesting “coding kata” by Brett Schuchert on the Object Mentor blog. The trick of a kata is that you grow the program step-by-step using tests, just like a kata in karate is tought to a student. The problem of this kata was the Shunting Yard algorithm of Dijkstra. I wanted to see if I could implement this kata in Scala.

Instead of writing the algorithm as described, the trick in a coding kata is that you first write a test, and then make the test pass. And subsequently each time adding a test and keeping it all in the green. Brett described the test-cases in his blog entry.

The Shunting Yard algorithm is used to convert a mathematical function in infix notation to a reverse polish or postfix notation. For instance, it can convert an expression like 3+4 to 3 4 +.

In order to implement this algorithm, one needs to do string parsing to break up the infix string. Scala has the parser combinators that can do just this. Using a parser I build an Abstract Syntax Tree (AST) which is a representation of the formula in a tree form. An AST for 3 + 4 and (3 + 4) * 4 looks like:


The AST representation of the formula can then easily be printed in a reverse polish notation, as the code shows. The final Parser looks like this:

import scala.util.parsing.combinator.syntactical._

abstract class Expr {
  def rpn:String
}
case class BinaryOperator(lhs:Expr, op:String, rhs:Expr) extends Expr {
	def rpn:String = lhs.rpn + " " + rhs.rpn + " " + op
}
case class Number(v:String) extends Expr { def rpn:String = v }
case class Variable(v:String) extends Expr { def rpn:String = v }
case class Function(f:String, e:List[Expr]) extends Expr { def rpn:String = {
	var s = ""
	e.foreach { x => s += x.rpn + " " }
	s += f
	return s
  }
}
object ShuntingYard extends StandardTokenParsers {
    lexical.delimiters ++= List("+","-","*","/", "^","(",")",",")

    def value  arser[Expr] = numericLit ^^ { s => Number(s) }
    def variable:Parser[Expr] =  ident ^^ { s => Variable(s) }
    def parens:Parser[Expr] = "(" ~> expr <~ ")"

    def argument:Parser[Expr] = expr <~ (","?)
    def func:Parser[Expr] = ( ident ~ "(" ~ (argument+) ~ ")" ^^ { case f ~ _ ~ e ~ _ => Function(f, e) })

    def term = (value | parens | func | variable)

    // Needed to define recursive because ^ is right-associative
    def pow  arser[Expr] = ( term ~ "^" ~ pow ^^ {case left ~ _ ~ right => BinaryOperator(left, "^", right) }|
    			term)
    def factor = pow * ("*" ^^^ { (left:Expr, right:Expr) => BinaryOperator(left, "*", right) } |
                        "/" ^^^ { (left:Expr, right:Expr) => BinaryOperator(left, "/", right) } )
    def sum =  factor * ("+" ^^^ { (left:Expr, right:Expr) => BinaryOperator(left, "+", right) } |
    					"-" ^^^ { (left:Expr, right:Expr) => BinaryOperator(left, "-", right) } )
    def expr = ( sum | term )

    def parse(s:String) = {
        val tokens = new lexical.Scanner(s)
        phrase(expr)(tokens)
    }

    def shunt(exprstr: String) : String = exprstr match {
      case null => return ""
      case "" => return ""
      case _ =>
    	parse(exprstr) match {
            case Success(tree, _) =>
                println("Tree: "+tree)
                val v = tree.rpn
                println("RPN: "+v)
                return v
            case e: NoSuccess => Console.err.println(e)
            	return e.toString
        }
    }
}

Of course I tested this using the following unit test, of which I will show only a small part, one can easily complete this by looking at Brett’s examples:

import org.junit.Test
import org.junit.Assert._

class ShuntingYardTest {
  @Test
  def test_11() {
    assertEquals("4 g f", ShuntingYard.shunt("f(g(4))"))
  }

  @Test
  def test_12() {
    assertEquals("3 4 19 f", ShuntingYard.shunt("f(3, 4, 19)"))
  }

  @Test
  def test_13() {
    assertEquals("3 4 2 * 1 5 - 2 3 ^ ^ / +", ShuntingYard.shunt("3 + 4 * 2 / ( 1 - 5 ) ^ 2 ^ 3"))
  }

  @Test
  def test_14 () {
    assertEquals("4 5 + 1 a 2 ^ + 8 b + 10 * f", ShuntingYard.shunt("f(4+5,1+a^2,(8+b)*10)"))
  }
}

Of all the tests that Brett defined, only one doesn’t pass, can you spot which one? Of course as this is one of my first exercises in Scala, it probably isn’t optimal yet, any improvements are welcome in the comments!



]]> http://blog.xebia.com/2009/07/02/shuntingyard-algorithm-in-scala/feed/ 0 http://blog.xebia.com/2008/09/15/beware-of-transitive-dependencies-for-they-can-be-old-and-leaky/ http://blog.xebia.com/2008/09/15/beware-of-transitive-dependencies-for-they-can-be-old-and-leaky/#comments Mon, 15 Sep 2008 11:54:15 +0000 Jeroen van Erp http://blog.xebia.com/?p=746 In many JEE apps today, you almost cannot forgo XML. Whether it is in configuration, data structures or service interfaces, you will certainly use a number of XML files. In a recent project we had to deal with a number of external services which used an XML interface. Little did we know that we introduced a potential time-bomb in our application…

Each of the accessed XML services in our application provided data for the same domain. So instead of writing or generating a great number of Java classes, we decided to use XSLT to transform the XML into a common format we could easily unmarshall. During performance and stress tests, this solution performed very well and showed no signs of degradation.

However…. Once we hit production, the application went OutOfMemory after a few days on a heap of 1GB. This was bad, how come our performance and stress tests didn’t show this? We decided to do the only sensible thing, and create a heapdump as soon as the application showed signs of stress. We started monitoring the java process using the jstat utility found in the JDK.

$ jstat -gcutil <processid> 1s 10000
  S0     S1     E      O      P     YGC     YGCT    FGC    FGCT     GCT
  0.00   0.00  99.98 100.00  26.66   2896  119.886  6000 32234.945 32354.831
  0.00   0.00 100.00 100.00  26.66   2896  119.886  6001 32234.945 32354.831
  0.00   0.00 100.00 100.00  26.66   2896  119.886  6001 32234.945 32354.831
  0.00   0.00 100.00 100.00  26.66   2896  119.886  6001 32234.945 32354.831
  0.00   0.00 100.00 100.00  26.66   2896  119.886  6001 32234.945 32354.831
  0.00   0.00 100.00 100.00  26.66   2896  119.886  6001 32234.945 32354.831
  0.00   0.00 100.00 100.00  26.66   2896  119.886  6001 32234.945 32354.831
  0.00   0.00 100.00 100.00  26.66   2896  119.886  6002 32240.277 32360.163
 

As soon as we saw the old space utilization hitting 100% and the Full Garbage Collector being unable to clean things up, we created a heapdump

$ jmap -dump:format=b,file=/tmp/heap.hprof <processid>

Using the Eclipse Memory Analyzer we looked into the heap dump, to see what was causing our memory to grow. This generated the following picture.



Clicking on the “Leak Hunter” link in the analyzer, quickly showed the possible culprit. The XMLReaderManager class from Xalan somehow had taken up a big chunk of available memory.



Searching through our Maven2 pom files showed no explicit dependency upon Xalan, but the dependency graph in the generated maven site showed it as a transitive dependency. One of our dependencies had xalan-2.6.0 noted down as a dependency. Googling around for this quickly showed us that there were two bugs against xalan, which described this exact problem: XALANJ-2178 and XALANJ-2195.

Looking at the source code for Xalan 2.6.0 we could indeed clearly notice the memory leak, whereas in 2.7.0 they updated it to prevent the class from keeping references around for too long. But why didn’t our stress tests on a similar machine and configuration show this?

The answer for this can also be found in the XMLReaderManager. It tries to cache the XMLReader in a ThreadLocal, so that it can give the same XMLReader to the Thread if it requests a new one. However in a JEE system, you normally have threadpools which scale up and down depending on system load at a given time. In our stress tests, we only simulated full loads for a prolonged time, and with this load, the XMLReaders weren’t leaked, as threads were being reused from the ThreadPool. If we had simulated a more varying load over a larger amount of time, we would have seen this occurring before we hit production.



]]> http://blog.xebia.com/2008/09/15/beware-of-transitive-dependencies-for-they-can-be-old-and-leaky/feed/ 3 http://blog.xebia.com/2008/05/08/javaone-2008-day-two/ http://blog.xebia.com/2008/05/08/javaone-2008-day-two/#comments Thu, 08 May 2008 08:31:16 +0000 Jeroen van Erp http://blog.xebia.com/?p=550 Today was the second day of the JavaOne 2008. Besides doing a lot of chatting in the JavaOne pavillion, and visiting all the cool parties this night, we also went to a number of sessions. Also today the NLJug had the James Gosling meeting we won for being the biggest JUG out here. After a long day of work, we finally had time to relax at the Adobe party and at the SDN party.

Todays topics included:

• Closures
• JavaFx, Groovy and Google Android
• Swing GUI testing
• Scripting

Mischa on part of his day @JavaOne
Closures Cookbook
Neal Gafter showed in his presentation a bunch of examples that should simplify coding and minimize use of boilerplate code using the “BGGA closure” proposal.

There are a few examples in which I would love to use closures for example like:

Adding a listener to a swing component.

ItemSelectable = ...;
is.addItemListener( { ItemEvent e => doSomething(e,is); });

or Aggregate Operations like ie. getting the highest Gpa from a list of Students.

final double highestGpa =
  students
  .filter({ Student s => s.graduationYear==THIS_YEAR })
  .map({ Student s => s.getGpa() })
  .max();

Though the impact of the BGGA closure proposal seems to be massive and the usage could result in very complex code, especially when combining closures with generics.

ie.

void for eachEntry(Map m, { K, V ==> void throws X } block) {
    for (Map.Entry e : m.entrySet()) {
        block.invoke(e.getKey(), e.getValue());
    }
}

All considered, it will result in cleaner code. But I am not convinced yet.

Going Mobile with JavaFx Script Technology, Groovy, and Google Android
Mike Mannion and Dierk König of Canoo shared there experiences on the JavaFx, Groovy and Google android.
Their experiences with the development of JavaFx were both negative and positive. Negative points were mainly about the development tools and the different syntax compared to Java.
On the other hand due to the declarative approach, the learning curve is quite low and productivity high.

Their experiences with Google android are quite positive. It has no real GUI restrictions, includes a lot of standard functionality and can be easily extended with Java libraries. The only
downside they found was the xml based GUI layout. Though the learning curve is quite steap.

Swing GUI testing made Easy
Testing GUIs can be difficult. This is due to a couple of things:

• Test should be automated, but GUIs are designed for humans
• Guis are usually composed of more then one class
• The number GUI interactions can be huge
• Conventional test coverage is not enough to cover all user interaction scenarios.

For testing Swing GUIs you have to meet a couple of requirements:

• The ability to simulate user input
• The need to have a reliable mechanism for finding GUI components..
• The ability to tolerate changes in position, size, layout etc..

Fest is an open source library that gives you these abilities. It has a good API, supports component lookup by name and type, integrates with JUnit and TestNG.

What is really nice is that Fest can automatically embed a screenshot of the desktop so you can see what the GUI looked like when the test failed.

Erik Jan on Web Beans
Web Beans is a framework that Gavin King wrote to be able to bind all kinds of java objects together. It came out of Seam but developed further there is a JSR for it. Oracle also has a similar solution for it.

Web Beans aims to be able to use ‘loose coupling with strong typing’. To be able to do this it has to use anotations in a special way. A simple example of a Webbean:

public
@Component
class Hello {
   public String hello(String name) {
      return “hello “ + name;
   }
}

A simple client

public
@Component
class Printer {
   @Current Hello hello;
   public void hello() {
      System.out.println( hello.hello(“world”) );
   }
}

The Web Beans container will now look for a component that is a Hello object and inject it it can also use constructor injection or an initializer injection with a parameter. But what if we have two implementations of the same interface, for instance one for testing and one for production. For this particular case Web Beans will have the ability to specify deployment types, this is done in a xml document.

It will also contain a way to add scope to beans and he created an extension on the EJB 3 listeners and a lot more. There will even be Spring support when it’s done… so take a look at his blog to find out what the status is…

Jeroen on Scripting
I had the privilege today of being in the close presence of some of the scripting gurus of the JVM. Today the scripting language shootout, the “Script Bowl” was being held. The format was an American Idol style format consisting of 3 rounds, the competitors were:

• Guillaume LaForge (Grails)
• Charles Nutter (JRuby)
• Frank Wierzbecki (Jython)
• Jorge Ortiz (Scala)

In the first round all the script gurus had to show a Twitter client in the scripting language of their choice and browse through the code. The richest clients were written in Groovy and JRuby, however Charles admittedly did not write his client himself. Jython and JRuby fell behind but they had working solutions. The jury, consisting of amongst others Ola Bini, considered Groovy the best, with a JRuby a runner up. The crowd sent the highest marks to JRuby followed by Groovy.

The second assignment was to build a web app based on the sample MySQL database. Guillaume used the Grails framework, whereas Charles presented a Rails application. Frank however didn’t implement all the functionality using Jython, and Jorge failed to implement anything at all and consequently showed us something else in Scala.Here highest marks were agin given to JRuby and Groovy.

The third round was the “Show your coolness” round. Guillaume showed us a HTML page from the JBoss Seam project which contained the documentation for the Seam and Groovy integration. Charles however showed 2 brilliant demos with cool graphical effects! The app he showed was called “A face for Stephen Hawking”, Google that if you’re interested. Here Jorge showed us how easy it is to write multithreaded applications using Scala. Using the actor keyword it is easy to spawn off new threads which share no mutable state whatsoever. The end result was that Charles won the shootout with JRuby, but Guillaume was a close second with Groovy.

If you want to follow a more blow by blow report, try following my Twitter stream. You can find me at: http://www.twiter.com/hierynomus.

Marco on his day @JavaOne
Oracle keynote
Actually I had planned to skip this one to catch some extra z’s but I just couldn’t bear to leave my companions in the cold by having them walk the dangerous streets of San Francisco all by themselves. We were treated to a nice stage play which reminded me of a tell sell commercial. “Wow Mike, so you say that if I process these orders those cool 3d bars will be updated right away?” Their product seemed cool though, worth taking a closer look at. If only I could remember it’s name ..

Creating a Java Platform, Enterprise Edition (Java EE Platform) Appliance, Using the GlassFish, OpenSolaris and Indiana Projects
Somewhere between naming this session and actually presenting it, the speakers probably decided they didn’t want to run the risk of a lawsuit, as the name ‘Indiana’ wasn’t mentioned even once. A good thing too, ‘coz mr. Jones still knows how to use that whip! This was about running web applications on appliances, using ZFS (a new kind of openSolaris file system) as a means to rollback, so there won’t be any need for hard-to-do reinstallations. A cool idea, but somehow I think I’ve heard it all before. Also, they used GlassFish v2 for this, and kept on saying how unfortunate it was that it wasn’t modular, Ehm, switch to the new version?

Open-Source Service-Oriented Architecture with Service Component Architecture and Apache Tuscany
Presented by somebody who looked a lot like a young Martin Sheen and the follow-up to a session I visited yesterday, this managed to make me even more interested in the topic at hand. For those not in the know: SCA is a development & deployment tool for SOA, based around assembled components called composites, and it may very well become the next big thing in the world of SOA. Or maybe it already is, who can keep track?

It’s all about the SOA: RESTful Service-Oriented Architecture at Overstock.com
Confession time: I had enrolled for an entirely different session (Java Persistence 2.0) but at the last minute decided to jump ship in what may well be the brightest moment I had all week. My reward was a nice story about how one of the biggest online retailers (it says so in the abstract, so it must be true) made the change from dreadful C to the wonderful delight that is Java in a very short period of time. Glad to know that after all that hard work they can finally enjoy some REST now.

Nothing? Wow, tough crowd ..

Creating a Compelling User Experience
As all Dutch will probably agree, SOA is nothing to laugh about, which is why this session served as a welcome relief. By far the most fun I had up until now it also gave me some fresh insight on interaction designers and the delicate relationship they hold with those people that just want things to look pretty. Two thumbs and a big toe up for this one.

JSR 303: From a World of Constraints to Constrain the World
Another non-SOA session, this one took me back to when I used the Struts Validation Framework to model the business logic of a large insurance company, resulting in several huge xml files. Ah, those were the days .. No big surprises here, which kind of disappointed me to be honest. Nice presentation though, even with the heavy French accent.

Epilogue
Some nice sessions, two of those that flew by, one which seemed to last forever (mental note: next time after gulping an entire can of soda go to the toilet first before attending a session). SCA was again the highlight of the day, but the presentation about user experience still gives me chuckles thinking about it. Anyway, halfway there, let’s see what tomorrow will bring.



]]> http://blog.xebia.com/2008/05/08/javaone-2008-day-two/feed/ 1 http://blog.xebia.com/2008/03/31/datetime-and-timezone-pains/ http://blog.xebia.com/2008/03/31/datetime-and-timezone-pains/#comments Mon, 31 Mar 2008 18:46:36 +0000 Jeroen van Erp date nuisances peculiarities calendar util interpretation surprise stored joda time datetime http://blog.xebia.com/2008/03/31/datetime-and-timezone-pains/ For as long as Java has been around, java.util.Date and java.util.Calendar have been nuisances. This will hopefully very soon be a thing of the past with the addition of JSR-310, the Date and Time API, to the Java API. At the basis of JSR-310 lies the Joda time library, which has been around for quite some time as a replacement for the standard Date and Calendar classes. However that this API is not without its own peculiarities need not come as a surprise, given the complexity of the human interpretation of time all over the world.


In our current project we have a value in the database that contains a timestamp in the UTC or “Zulu” timezone. This timestamp is fetched from the database using a java.sql.Timestamp instance. Our application however deals with org.joda.time.DateTime instances in the local time zone, which for the Netherlands are either CET (UTC+0100) or CEST (UTC+0200) depending on Daylight Savings Time.

So in a nutshell our problem encompasses converting a UTC java.sql.Timestamp to an org.joda.time.DateTime in the correct time zone, which represents the correct time. Let’s start off with writing a small skeleton JUnit test:

public class DateTimePainsTest extends TestCase {

    private static final String WRONG_TIME = "2008-03-26T20:13:39.059+01:00";
    private static final String CORRECT_TIME = "2008-03-26T21:13:39.059+01:00";
    private long millis;
    private Timestamp timestamp;

    protected void setUp() throws Exception {
        millis = new Date(1206558819059L).getTime(); // 2008-03-26T20:13:39.059+00:00
        timestamp = new Timestamp(millis);
    }

    public void testMillisAndTimestamp() {
        assertEquals(millis, timestamp.getTime());
        assertEquals("2008-03-26 20:13:39.059", timestamp.toString());
    }
}

Now we want to convert this timestamp to a DateTime object representing the following string: 2008-03-26T21:13:39.059+01:00. We first try the most simple option we can think of:

public void testDateTimeConversion1() {
        DateTime dateTime = new DateTime(timestamp.getTime(),
                DateTimeZone.forOffsetHours(1));
        assertEquals(CORRECT_TIME, dateTime.toString());
}

This test fails. In order to make the test succeed, we have to expect WRONG_TIME in there. Though according to the JavaDoc of DateTime this should have been a step in the right direction:
DateTime(long instant, DateTimeZone zone)
Constructs an instance set to the milliseconds from 1970-01-01T00:00:00Z using ISOChronology in the specified time zone.
Maybe we should first force the time zone to UTC before converting the DateTime to a European time zone. Let’s give that a try:

public void testDateTimeConversion2() {
        DateTime dateTime = new DateTime(timestamp.getTime(), DateTimeZone.UTC)
                .withZone(DateTimeZone.forOffsetHours(1));
        assertEquals(CORRECT_TIME, dateTime.toString());
}

This test also fails, substituting WRONG_TIME for CORRECT_TIME yields a green bar, but an undesired result, so we should take a different approach. LocalDateTime is a Joda time class which does not take into account any time zone information. May we can convert this to a DateTime in the correct time zone. Let’s write a new test case:

public void testDateTimeConversion3() {
        DateTime dateTime = new LocalDateTime(timestamp)
                .toDateTime(DateTimeZone.forOffsetHours(1));
        assertEquals(CORRECT_TIME, dateTime.toString());
}

Again, this test fails. We need to expect the WRONG_TIME in there to get a green bar. What seems to do the trick is converting the LocalDateTime to a DateTime in the UTC time zone, and then assigning the correct time zone to that. Lo and behold, the following test case works:

public void testDateTimeConversion4() {
        DateTime try3 = new LocalDateTime(timestamp).toDateTime(DateTimeZone.UTC)
                .withZone(DateTimeZone.forOffsetHours(1));
        assertEquals(CORRECT_TIME, try3.toString());
}

Though this solution works, I think it is not the most optimal solution. Does anyone know of a better solution? If so, please add a comment.



]]> http://blog.xebia.com/2008/03/31/datetime-and-timezone-pains/feed/ 1 http://blog.xebia.com/2008/03/27/maven2-dashboard-plugin-released/ http://blog.xebia.com/2008/03/27/maven2-dashboard-plugin-released/#comments Wed, 26 Mar 2008 22:28:38 +0000 Jeroen van Erp dashboard maven2 released plugin quality hefty buildsystem metrics http://blog.xebia.com/2008/03/27/maven2-dashboard-plugin-released/ Quality is an everyday part of the life of a Xebia software developer. One of the ways to get insight into quality is by looking at metrics like FindBugs, PMD, Simian, Code Coverage, etc. With large software products consisting of different modules, quality assurance can become quite a trying task. This means that tools which alleviate this burden are a welcome addition to our toolbox.


The Maven 1.x dashboard is such a tool. The Maven site is integrated into our continuous integration process, and the dashboard that is generated provides a birds-eye view of a lot of our quality metrics. This is especially useful when dealing with multi-module projects. For the Maven2 build system a similar plugin is currently available on Codehaus. However this plugin is not final yet and in the past we had some trouble to get it to work. Therefore we decided to try and build our own plugin.

As our toolbox heavily leans on a lot of open source products, we decided to give something back to the community and release this plugin as an open source product. The dashboard plugin can be found at the following URL: http://mojo.os.xebia.com/maven-dashboard-plugin. On the site you can find many configuration examples.

In order to use the plugin in your pom, you’ll have to do some basic configuration. The following example shows what you need to add to your pom.xml

  ...


      xebia-maven2
      Xebia Maven2 Repository
      http://mojo.os.xebia.com/repository
      
  
  ....
  

      ...

        com.xebia.mojo
        maven-dashboard-plugin
      
    
  

This configuration will generate a full dashboard when you run the build using the following command line:

mvn site dashboard:dashboard site:deploy

The result of the execution of this command for the maven dashboard plugin itself looks like this


The source code for the plugin is released in Subversion, and the Jira issuetracker can be found here.

Now that we’ve got a Dashboard for Maven2 projects, we can finally get the quality overview and focus on fixing quality assurance issues instead of finding them throughout the different modules of our projects.



]]> http://blog.xebia.com/2008/03/27/maven2-dashboard-plugin-released/feed/ 7 http://blog.xebia.com/2008/03/13/javaone-2008-scripting-preview/ http://blog.xebia.com/2008/03/13/javaone-2008-scripting-preview/#comments Thu, 13 Mar 2008 19:45:49 +0000 Jeroen van Erp scripting panel languages javaone groovy javascript language ruby http://blog.xebia.com/2008/03/13/javaone-2008-scripting-preview/ The biggest Java event of the year is coming up again, the JavaOne. For me this is a place of inspiration. Seeing the newest technologies in action and talking to some of the great minds in the Java world, how can one not become inspired!


One of the big subjects this year, which has been getting bigger over the past few years is the scripting support on the Java Platform. There are numerous talks on all the different scripting languages. Let’s get some numbers for an overview:

	Technical Session	Panel Session	Birds of a Feather
Scala	1		1
Groovy / Grails	7	1	4
(J)Ruby / Rails	7	1	3
Jython	1		1

These numbers are based on whether the subject contains the scripting language, or whether the presenters show examples in the language. One of the languages I didn’t mention is the JavaScript language. JavaScript is a real scripting language, which is supported on the Java platform, but almost every talk on some Webby subject intrinsically deals with JavaScript. This would severely pollute the count.

What we can see from the above table is that Groovy and (J)Ruby are almost equally represented. Whether the organization did this on purpose, or whether it is purely coincidental is unknown to us of course. What is noticeable though is that just one BOF in which Groovy examples are used is presented by a Sun employee, whereas many Technical Sessions on (J)Ruby are presented by Sun employees.

At the JavaOne we’ll see what direction the community will take with regards to the scripting languages on the JVM. Especially the panel session on Wednesday will provide some insight in how the different languages relate and how the community regards them. I’ll be there during the panel session to hear and see what the community deems the ideal scripting language.



]]> http://blog.xebia.com/2008/03/13/javaone-2008-scripting-preview/feed/ 2 http://blog.xebia.com/2007/10/04/leaking-memory-in-java/ http://blog.xebia.com/2007/10/04/leaking-memory-in-java/#comments Thu, 04 Oct 2007 10:20:17 +0000 Jeroen van Erp testgc beginindex endindex substring offset substrings getsubstring char http://blog.xebia.com/2007/10/04/leaking-memory-in-java/ Don’t we all remember the days when we programmed C or C++? You had to use new and delete to explicitly create and remove objects. Sometimes you even had to malloc() an amount of memory. With all these constructs you had to take special care that you cleaned up afterwards, else you were leaking memory.


Now however, in the days of Java, most people aren’t that concerned with memory leaks anymore. The common line of thought is that the Java Garbage Collector will take care of cleaning up behind you. This is of course totally true in all normal cases. But sometimes, the Garbage Collector can’t clean up, because you still have a reference, even though you didn’t know that.

I stumbled across this small program while reading JavaPedia, which clearly shows that Java is also capable of inadvertent memory leaks.

public class TestGC {
  private String large = new String(new char[100000]);

  public String getSubString() {
    return this.large.substring(0,2);
  }

  public static void main(String[] args) {
    ArrayList subStrings = new ArrayList();
    for (int i = 0; i < 1000000; i++) {
      TestGC testGC = new TestGC();
      subStrings.add(testGC.getSubString());
    }
  }
}

Now, if you run this, you'll see that it crashes with something like the following stacktrace:

Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
at java.lang.String.(String.java:174)
at TestGC.(TestGC.java:4)
at TestGC.main(TestGC.java:13)


Why does this happen? We should only be storing 1,000,000 Strings of length 2 right? That would amount to about 40Mb, which should fit in the PermGen space easily. So what happened here? Let's have a look at the substring method in the String class.

public class String {
  // Package private constructor which shares value array for speed.
  String(int offset, int count, char value[]) {
    this.value = value;
    this.offset = offset;
    this.count = count;
  }

  public String substring(int beginIndex, int endIndex) {
    if (beginIndex < 0) {
      throw new StringIndexOutOfBoundsException(beginIndex);
    }
    if (endIndex > count) {
      throw new StringIndexOutOfBoundsException(endIndex);
    }
    if (beginIndex > endIndex) {
      throw new StringIndexOutOfBoundsException(endIndex - beginIndex);
    }
    return ((beginIndex == 0) && (endIndex == count)) ? this :
      new String(offset + beginIndex, endIndex - beginIndex, value);
  }

We see that the substring call creates a new String using the given package protected constructor. And the one liner comment immediately shows what the problem is. The character array is shared with the large string. So instead of storing very small substrings, we were storing the large string every time, but with a different offset and length.

This problem extends to other operations, like String.split() and . The problem can be easily avoided by adapting the program as follows:

public class TestGC {
  private String large = new String(new char[100000]);

  public String getSubString() {
    return new String(this.large.substring(0,2)); // <-- fixes leak!
  }

  public static void main(String[] args) {
    ArrayList subStrings = new ArrayList();
    for (int i = 0; i < 1000000; i++) {
      TestGC testGC = new TestGC();
      subStrings.add(testGC.getSubString());
    }
  }
}

I have many times heard, and also shared this opinion that the String copy constructor is useless and causes problems with not interning Strings. But in this case, it seems to have a right of existence, as it effectively trims the character array, and keeps us from keeping a reference to the very large String.



]]> http://blog.xebia.com/2007/10/04/leaking-memory-in-java/feed/ 10 http://blog.xebia.com/2007/09/26/testing-without-aspects/ http://blog.xebia.com/2007/09/26/testing-without-aspects/#comments Wed, 26 Sep 2007 20:59:04 +0000 Jeroen van Erp thief jail victim itemsinposession steal putinjail policeagentaspect 1000 http://blog.xebia.com/2007/09/26/testing-without-aspects/ Recently I wanted to add an aspect to some domain object, so that it was saved, the moment it changed state. However, after adding this aspect, the whole build of course failed, because a lot of the unit tests weren’t expecting the calls which were now woven into the domain object.


Of course I could alter all the unit tests so that they reflected the change in code. But this seems to defy the whole purpose of the aspect. Another option is of course postponing the weaving until after the unit tests. This didn’t seem to be the correct solution either, because I did want to write at least some tests to test that the aspect was doing its job.

So how did I get around this? The answer lay in Spring AOP. It shares much of the syntax of AspectJ, though it doesn’t (yet) offer all of the functionality. I’ve created a small sample for your reading pleasure.

At first we’ll look at a simple domain without aspects.

Victim.java

public class Victim {
    private int itemsInPosession;

    public Victim(int itemsInPosession) {
        this.itemsInPosession = itemsInPosession;
    }

    public void itemStolen() {
        if (itemsInPosession > 0) {
            itemsInPosession--;
        }
    }

    public int getItemsInPosession() {
        return itemsInPosession;
    }
}

And Thief.java

public class Thief {
    public void steal(Victim victim) {
        victim.itemStolen();
    }
}

Of course we have a unit test to test that the Thief indeed steals from a Victim:

public class ThievingTest extends TestCase {
    public void testShouldSucceedInStealing() {
        Victim victim = new Victim(1000);
        Thief thief = new Thief();
        thief.steal(victim);
        assertEquals(999, victim.getItemsInPosession());
    }
}

Running this test shows that the Thief succeeds in stealing an item from the Victim. But of course, we can’t let Thiefs randomly steal from Victims. We need to introduce a PoliceAgent who can put the Thief in Jail. Let’s add an interface to Jail, and write a new test asserting that the Thief is put in Jail for stealing.

Jail.java

public interface Jail {
    public void putInJail(Thief thief);
}

And the second unit test class:

public class ThievingJailTest extends TestCase {
    private Jail jail;

    public void setUp() {
        jail = EasyMock.createMock(Jail.class);
    }

    public void testShouldLandInJailForStealing() {
        Victim victim = new Victim(1000);
        Thief thief = new Thief();
        jail.putInJail(thief);
        EasyMock.replay(jail);
        thief.steal(victim);
        EasyMock.verify(jail);
        assertEquals(1000, victim.getItemsInPossession());
    }
}

Of course, this test fails, because there is no way to put the Thief in Jail yet. Let’s add the PoliceAgent as an aspect, because we want to intercept the Thief that is stealing:

PoliceAgentAspect.java

@Aspect
public class PoliceAgentAspect {
    private Jail jail;

    @Around("execution(void *Thief.steal(..))")
    public void arrestThief(ProceedingJoinPoint joinPoint) {
        Thief thief = (Thief) joinPoint.getThis();
        jail.putInJail(thief);
    }

    public void setJail(Jail jail) {
        this.jail = jail;
    }
}

In order for Spring AOP to notice the aspect, and do the wiring, we need to add an applicationContext.xml with the correct beans. In this simple example, the following suffices (I’ve stripped the namespace declarations for readability):

You can see that in the Spring context I’ve mocked the Jail. This code previously was in the unit test. Also I’ve added the Thief as a bean which can be injected. This ensures that Spring can intercept the Thief with the PoliceAgentAspect. The modified test looks like this:

public class ThievingJailTest extends AbstractDependencyInjectionSpringContextTests {
    private Jail jail;
    private Thief thief;

    @Override
    protected String[] getConfigLocations() {
        setAutowireMode(AUTOWIRE_BY_NAME);
        return new String[] {"applicationContext.xml"};
    }

    public void testShouldBePutInJailByPoliceAgentAspect() {
        Victim v = new Victim(1000);
        jail.putInJail(thief);

        EasyMock.replay(jail);
        thief.steal(v);
        EasyMock.verify(jail);
        assertEquals(1000, v.getItemsInPosession());
    }

    public void setJail(Jail jail) { this.jail = jail; }
    public void setThief(Thief thief) { this.thief = thief; }

When we now run this test it succeeds. Also our old test still runs perfectly and provides a green light. Using Spring AOP, we can ensure that our code works when not using aspects, and that the adding of aspects to our code gives the intended results.



]]> http://blog.xebia.com/2007/09/26/testing-without-aspects/feed/ 5