With all the big-data postings, now something about traditional SQL, running at DB2 9. We had created a web-application, that was basically a view for a database. The application displayed content of a database after the user would enter search parameters, or everything when no parameter was entered. The database contained a few hundred-thousands records. All in all very simple.
The software worked fine at our test-environment. But when we deployed the software into production, it would hardly ever display data. In the logs we found frequent locking errors as follow:

15:23 ERROR (org.hibernate.engine.jdbc.spi.SqlExceptionHelper) – DB2 SQL Error: SQLCODE=-911, SQLSTATE=40001, SQLERRMC=68, DRIVER=4.1

Searching the web revealed what that error meant: a deadlock or timeout has occurred. We knew the same database was also accessed by another application, Activiti running processes written in BPMN. The processes in the Activiti-engine only inserted a few records per minute, so we didn’t expect that application having a big influence. Yet how can a few inserts per minute have such a big impact?
After some talking to a DBA’er and doing our own experimentation we found out about the problem, which will describe here.

 Read more

One of the domains for which data lends itself well to be represented as a graph is trade. We can take any tradable good, represent the trading actors as nodes, and represent the (amount of) traded goods as (properties of) edges. As an example, the graph below displays the international trade of fish in 1998 (source data), where the nodes represent countries, the sizes of nodes represent the total exports per country, and the edges represent the fact that the edge’s source country exported a given amount of fish to the destination country:

 Read more

In this blog I will show a way to do performance testing with Selenium. The reason I use Selenium for performance testing is that some applications use proprietary protocols between the application layer in the browser and the server.

So just capturing the traffic between the server and replaying modified traffic is not that simple.

An example is testing GWT applications. In a previous blog I wrote why this is difficult.

To create a test script in Selenium the first thing I do is record a test with Selenium IDE
After recording a script I export the script to JUnit3 (Remote Control). This will generate a JUnit test script which can be run to test the application.

The next thing you need is a solution to run a lot of JUnit test cases at the same moment.

Here you see a visual representation of the whole test chain.

 Read more

Active Directory is Microsoft’s implementation of LDAP. LDAP is often used for user authentication and authorization. There are options to modify the LDAP directly from Java without the Powershell. With Spring-LDAP in combination with Spring-ODM you can read, write and query the AD (ActiveDirectory). We ‘developers’ like it, but operations don’t.

Using such an API is low-level and you have to make sure when manipulating the AD you set the right fields and attributes:

• You have to know the required fields (unicodePwd, instead of userPassword)
• You must understand the encoded fields like userAccountControl, password in bytes
• You as developer are responsible for the interaction, and you can screw the AD.
• You must fix problems when operations may change, update or upgrade AD.

Most important in our case is that operations are the owners of AD and responsible for it. They strongly suggested to use the Powershell when modifying user accounts. The Powershell is running on the Domain Controller where AD is located and can be used as an API. It only runs on Windows, but we want to host our Java software on Linux, so we decided to communicate over SSH.

 Read more

Java 8 will be delivered in less than one year. One of the most impacting projects that will appear with this version certainly is the Lambda Project, referenced as JSR335 and supervised by Brian Goetz. This project aims to provide you an easier way to integrate functional programming and parallel computing. The goal is to improve the productivity of developers by enhancing the expressivity of the language and to optimize the performance of Java application by offering an easier way to exploit multi-core architectures. But what will be the impact on the Java community of the results of the Project Lambda?

 Read more

In my current role as application performance specialist I see a lot of different frameworks that are used to develop applications faster, to make a more reactive frontend application or just a framework that is the latest and greatest from a development point of view.

One of these frameworks is Google Web Toolkit. Frameworks like this are a good thing for development… but for performance testing they might be a problem.

For performance testing the best practice is to use a proxy server recording all traffic between a browser and the application. In most cases this is just plain http with html or xml as a protocol. So it is easy to replay the recorded test and parameterize it with different values. Read more

In this post, my colleague Barend showed how one can conditionally ignore certain tests in JUnit. In this post we will take a look at how this can be solved in TestNG, another popular testing framework.  Read more

A useful technique that I reinvent every once in a while is conditionally ignoring JUnit tests. Unit tests are supposed to be isolated, but occasionally you hit something that makes assumptions about the environment, such as code that executes a platform-specific shell command or (more commonly) an integration test that assumes the presence of a database. To keep such a test from breaking unsuspecting builds, you can @Ignore it, but that means you have to edit the code to run the test in a supported environment.

Proper Maven projects put their integration tests in a separate source folder called src/it/java and put an extra execution of the maven-surefire-plugin into their pom.xml, tied to the integration-test phase of the Maven build lifecycle. This is Maven’s recommended way of setting these up. It ties in beautifully with the pre-integration-test and post-integration-test phases that can be used to set up and tear down the environmental dependencies of the integration test suite, such as initializing a database to a known state. There is nothing wrong with this approach, but it’s a bit heavy handed for the simplest of cases.

In these simple situations it’s easier to just keep the integration tests in the src/test/java directory and run them along with all your other tests. However, you still need a way to trigger them only when the right environment is present. This is easily dealt with by writing your own JUnit TestRunner and some custom annotations, as shown below.

 Read more

Mongo — the document oriented NoSQL database supported by 10gen — offers a compact, easy to learn and well documented query language. Unfortunately, using Mongo with its Java driver can be tricky: querying, mapping results and handling polymorphism require lots of code. Some libraries aim to simplify this (like Morphia), but none allows to query in a shell fashion. Jongo tries to fill that need, querying with the use of strings and unmarshalling results into Java objects.

 Read more

Introduction

We all know the rule in some form or another: three times and you automate. And although I try to apply it, I find myself repeating some things with every new project, like creating a new CI server.

One solution would be to have a centralized server for all your projects, but customers are mostly not willing to depend on services outside of their network, which means you are often stuck with having to create a local solution for each project.

Having done this three times, I decided to start a repository to store scripts to automate development machine setup. One command to get you a fully functional system you only need to tweak to suit your needs. This blog will show you how to quickly set up a Jenkins server on a local virtual machine.

 Read more