Spring is a great framework, but I expect that some Spring based applications are subject to visibility problems (a type of concurrency problem). This blog entry describes the cause and effects of this problem, and also how it can be solved.

Visibility problems

Most programmers don't have much experience with multi threading and have a very simplistic view on reality: if a thread makes a change to a variable, this change is directly visible to all other threads that access the same variable. This view is called sequential consistency, but the problem is that no virtual machine automatically provides this view.

The reason for this apparently faulty behavior is performance. Most variables are not shared between threads, so no special instructions (memory barriers) have to be added to make the program sequentially consistent. If these instructions were automatically added to all variable accesses, it would greatly reduce performance because a lot of optimizations would be excluded from being used. To name a few:

1. usage of super fast local-memory (like caches and cpu-registers). It also could reduce the performance benefits of multi-core systems because the increased access to main memory would become the bottleneck.
2. reordering of instructions to increase the chance of a cache hit.

The consequence of a visibility problem, is that a value written to a variable by one thread, doesn't have to be visible in other threads (maybe never). This could lead to an easy to detect NullPointerException, but it can also lead to much harder to detect problems like shown in the following example:

public class SomeRunnable implements Runnable{
    private boolean stop = false;

    public void stop(){
        stop = true;
    }

    public void run(){
        while(!stop){
                System.out.println("hello");
        }
    }
}

There are a few reasons why this runnable could fail to stop:

1. stop is not safely published: the thread that executed the run method, doesn't need to see a change in the stop value. A possible scenario would be: thread1 (the thread that executed run) is running on cpu1 (with cache1). Thread2 (the thread that calls stop) is running on cpu2 (with cache2). When thread1 runs, it needs the value of stop, sees the initial false value and places it in his cache. When thread2 executes stop, stop is set to true. But it might happen that this value remains in cache2 for an undetermined time and isn't flushed to main memory, so thread1 never sees the new value when it happens to read from main memory. Even if the value is flushed to main memory, thread1 could still be reading the stale value from cache1.
2. because the access to the stop variable isn't safely published, and the value doesn't change in the loop, the 'compiler' (JIT, cpu etc) could decide to replace the variable read by the constant 'true' in that loop.

Visibility problems can lead to hard to detect errors and unpredictable behavior, and you definitely want to keep them out of your system. And although most cpu's are not subject to visibility problems (because of strong Memory Coherence), betting on this behavior is asking for problems (the application also wouldn't be platform independent).

Spring applications and visibility problems

I expect that a lot of Spring applications are subject to visibility problems. If you look at singleton beans for example (dao's, controllers, managers etc), these beans often are shared by many threads, to name a few:

• threads from the servlet container
• threads from remoting middleware
• threads from jmx
• threads from triggers or other internal threads

Take a look at the following example:

public class EmployeeManagerImpl implements EmployeeManager{

    private EmployeeDao employeeDao;

    public setEmployeeDao(EmployeeDao employeeDao){
        this.employeeDao = employeeDao;
    }

    public void fire(long employeeId){
        Employee employee = employeeDao.load(employeeId);
        employee.fire();
    }
}

And the bean configuration that belongs to it:

    <bean id="employeeManager" class="EmployeeManagerImpl">
        <property name="employeeDao" ref="employeeDao"/>
    </bean>

The visibility problem here, is that employeeDao, set by the thread that constructed the employeeManager, isn't safely published and doesn't have to be visible in other threads. If one of those threads calls the fire method on the EmployeeManagerImpl, it could lead to a NullPointerException because that thread still sees a null value for the employeeDao.

The problem can be solved in a few ways:

1. make employeeDao volatile. Although it sounds quite strange, because the employeeDao is not going to change after it has been set, volatile variables are always safely published. Personally I don't like this solution much because it is misleading (the value is never going to change after the object has been constructed) and it also reduces performance (the variable always has to be read from main memory instead of local memory).
2. make employeeDao final, because final variables also are safely published. The problem with final variables is that they only can be set by constructor based Dependency Injection (DI) and Spring promotes setter based DI. Personally I prefer constructor based DI, because it is a lot easier to guarantee class invariants. Although those setters only are visible in an implementing class, and not in the interface, I still don't feel happy about because it also makes classes harder to understand. But constructor based DI isn't perfect either; I guess most of us have struggled with large constructors.

Problem not that bad?

Before going into detail why the problems maybe are not that bad, I need to explain the new Memory Model found in Java 5 and higher (JSR-133) (see footnote). The model describes under what kinds of conditions a variable will be visible in other threads and it also shields you from reasoning about caches (they are not mentioned in the Java Memory Model).

The model is expressed in terms of actions:

1. read/writes to normal/volatile/final variables
2. lock acquire/release
3. thread start/join

The model also contains a set of happens-before rules between actions. If a happens-before rule applies between action1 and action2, then all changes made by action1 are visible in action2. Examples of happens-before rules are:

1. Program Order rule: each action in a thread happens-before every action in that thread that comes later in the program order. It is important to realize that normal variable reads/writes, can be reordered as long as they keep the 'within-thread as-if-serial semantics': the reordering should not be visible inside the thread.
2. Volatile variable rule: a write to a volatile variable happens-before every subsequent read of that same variable (that is why making the employeeDao volatile works).
3. Monitor lock rule: an unlock on a monitor lock happens-before every subsequent lock on that same monitor lock. (The same goes for the the new Lock found in Java 5.
4. Transitivity rule: if action1 happens-before action2, and action2 happens-before action3, then action1 happens-before action3.

If there is a happens before relation between the write and the read of a variable, then it is safely published.

Safe handoff

These happens-before rules can be used to check if there is a happens-before relation between two actions. Take a look at the following example:

int a=0;
volatile boolean b=0;

void initialize(){
        a=console.readInteger();
        b=console.readBoolean();
}

void print(){
        print(b);
        print(a);
}

A possible ordering of actions could be:

Because action1 happens-before action2 (program order) and because action2 happens-before action3 (volatile variable rule) and because action3 happens before action4 (program order), action1 happens before action4 (remember that the happens before rules are transitive). This means that the change in action1, is visible in action4. This technique is called 'safe handoff' (aka 'piggybacking on synchronization').
Safe handoff uses the safe publication of a variable X (in this case b), to safely publish all unsafely-published-variables that are changed before X (in this case a).

Safe handoff in Spring

The Spring applicationcontext also provides safe handoff (only in Java 5 and higher): after a singleton bean is created, it is placed in a singletonmap. When it is needed, it is retrieved from that map. But instead of using the 'volatile variable' rule, it uses the 'monitor lock' rule (action4 and action5)

The following table shows a very simplified ordering of actions (variable mapentry.ref is the variable 'X' from the 'save handoff' technique):

This means that there is a happens before relation between action2 and action8, so the value set in the employeeDao (action2), is visible when it is read by another thread (action8). That is why standard Spring singletons don't have visibility problems. If you want to see it for yourself, check out the DefaultSingletonBeanRegistry and the 'public Object getSingleton(String beanName, ObjectFactory singletonFactory)' method

Conclusion

I'm glad that standard singleton beans aren't subject to visibility problems (under Java 5 and higher), but I don't think it is the correct way to deal with concurrency control in Spring (or in general):

1. objects can't be safely used in a different environment. So your objects are tied to the (current version of the) Spring framework to make them work correct.
2. the save handoff behavior is not guaranteed by Spring, and it only works under Java 5 and higher. The behavior is undefined in older JVM's.
3. are beans with different scopes than singletons free from visibility problems?

I don't want to depict Spring as a bad product: it really is amazing and I'm always happy when I can use it on a project. But I do think that developers should be more careful when writing code that is used in a multi threaded environment.

A part of this responsibility should be taken bij developers themselves, if you write threadsafe code, you need to know what you are doing. But I also think the guys behind Spring should take part of this responsibility: not all developers have a lot of experience with writing multi-threaded code and I guess most don't know anything about the Java Memory Model. If these issues are not mentioned in the Spring documentation, a lot of developers will remain agnostic and this is going to lead to problems in the future.

Footnote:
If you want a more in depth explanation of the new Java Memory Model, I suggest 'Java Concurrency in Practice' from Brian Goetz, Tim Peierls, Joshua Bloch, Joseph Bowbeer, David Holmes and Doug Lea, or check out the following website The Java Memory Model

I want to thank colleagues, acquaintances and the guys from the concurrency mailing list for proofreading this post.

Filed under Concurrency Control, Java, Spring | 4 Comments »