Stopping a Thread is complicated. In the beginning the Thread.stop() method was added for this purpose, but is has been
deprecated for a long time. The reason is when some thread calls stop() on a victim thread, a ThreadDeath error is thrown inside the victim thread. And while this error propagates up the stacktrace, all locks the victim thread owns, are released. This means that the victim-thread could leave objects (on which it held locks) in an inconsistent state. So stopping a thread should be a cooperative mechanism between threads: the victim thread and the initiating thread both need to agree upon a protocol being used.

Luckily I don’t see the usage of Thread.stop() (or even worse, the Thread.destroy()) often, but the idiom displayed in the example below, is something I do see regularly. Although the exception handling is sometimes forgotten. This Task is executed by some thread, and when a different thread wants to stop this Task, it calls the stop method on the Task and the stop variable is set to true. As soon as the victim thread starts the next iteration of the loop, it reads the stop variable, ends the loop and returns from the run method (returning from the run method eventually terminates the victim thread).

public class Task implements Runnable{
	private boolean stop = false;

	public void stop(){
		stop = true;
	}

	public void run(){
		while(!stop){
			try{
				runSingleUnit();
			}catch(Exception ex){
				//log it, in most cases we
				//don't want to break the loop
			}
		}
	}

	public void runSingleUnit()throws Exception{
		System.out.println("executing some service");
	}
}

The problem is that this example is faulty. There are 2 reasons and they are both linked to the Java Memory Model (JMM) that after a long period is completely defined in Java 5 and higher (see the JSR 133 for more information):

1. visibility
2. reorderings

Visibility

Because the read and write of the stop variable are not special (so not volatile, not final, and not done inside a synchronized context) the JVM doesn’t provide any guarantee that the most recently written value is read (there are more ways to provide visibility guarantees but they are out of scope for this post). This means that the victim thread could see the initial written value till end of time. And this effectively transforms the loop to an infinitive one.

You might wonder why a thread is not able to see the most recently written value, because this most surely be some kind of error. Well… it isn’t because this desired behaviour, called sequential consistency, prevents a lot of optimizations from being used. There are different forms of local memory (multi level caches, cpu registers) where value’s can be stored instead of the main memory. This can result in the following scenario’s:

1. local memory doesn’t need to contain the most recently written value by another thread. So
   a thread that uses this local memory, doesn’t need to see writes made by other threads.
2. local memory contain the most recently written value made by the current thread without being visible in main memory, so a write doesn’t have to be visible to other reading threads.

The chance of this happening with the strong cache coherence most cpu’s provide (realized by snooping and sniffing caches for example) isn’t that big, but it certainly is possible. And concurrency problems have the tendency to appear only once in a while and therefore are very hard to reproduce (they could also be hardware dependent).

Reasoning about caches, invalidation, memory fences and the like can complicate the platform independent view Java developers have. Therefore the JMM is defined in terms of happens before rules where each rule defines a relation between 2 actions (read/write/volatile-read/volatile-write/lock-acquire/lock-release), and this should shield us from reasoning about these low level issues. If a happens before relation between 2 actions (in this cases the normal write and normal read) can be found, all changes made in first action (the write) are visible in second action (the read). In this case there is no such relation, so no guarantee can be given about the write of the stop variable in the stop method (executed by the initiating thread) ever being visible at the read of the stop variable in the run method (executed by the victim thread).

Reorderings

The compiler could decide to hoist the read of the stop variable out of the loop, a performance increasing technique called ‘Loop invariant code motion’

public void run(){
	if(stop)
		return;

	while(true){
		try{
			runSingleUnit();
		}catch(Exception ex){
			//log it
		}
	}
}

This optimization is possible because inside the loop the stop value is not changed, so why do an expensive memory access every time. As you can imagine, the new run method never ends as soon as it passes the if(stop) check. This reordering is allowed because from the victim thread point of view, the transformed code is equivalent to the original one, so the within-thread as-if-serial semantics of the transformed code is the same as the original.

Solution

The simplest way to solve these problems is to make the stop variable volatile. Volatile variables prevent:

1. visibility problems: there is a happens before relation (called volatile variable rule) between the write to a volatile variable and the subsequent read of the same variable, so a read will always see the most recently written value.
2. reordering problems: the instructions prior to the volatile read of the stop variable, are not allowed to be moved over this read. This prevents the movement (reordering) of the check outside the loop.

But using a read and write under a lock (should be the same lock) should also fix the problem:

public synchronized void stop(){
	stop = true;
}

private synchronized boolean isStopped(){
	return stop;
}

public void run(){
	while(!isStopped()){
		...
	}
}

The reason why the synchronized approach works:

1. visibility: there is a happens before relation between the write of stop and the read of stop (called monitor lock rule) that makes sure that all changes made before releasing the lock in the stop method, are visible when the lock is acquired in the isStopped() method.
2. reordering the movement of instructions prior to the lock acquire is not allowed to jump over the release of the lock. This principle is called the ‘Roach Motel’: instructions in front of the lock acquire are not allowed to jump over the lock release, but are allowed to jump over the lock.acquire. The same principle can be applied to instruction after the lock release: they are not allowed to jump over the lock acquire, but they are allowed to jump over the lock.release. Instruction that are between the lock acquire and the lock release are not permitted to jump over the lock.release or lock.acquire (a cockroach doesn’t want to leave his dark shelter).

This stopping technique is quite useful if the execution of a single iteration doesn’t take too much time. If it takes a longer time, different techniques need to be considered like thread interruption. For more information about stopping threads I recommend chapter 7 “Cancelling and Shutdown” from ‘Java Concurrency in Practice’. For more information about the JMM I recommend chapter 16 “The Java Memory Model” from the same book or check out the the JMM website.

So, without further ado, let’s start at the bottom: number 10.

Nested Monitor Lockout

It is very easy to cause liveness problems with conditions (aka condition variables) but it is not just the direct usage that can cause problems, also techniques like object composition can be quite dangerous.

A condition is a very low level synchronization structure that makes it possible for threads to wait for an event to occur, e.g. the return of a license in a semaphore or the placement of an item on a queue. So a lot of higher order concurrency structures are build on top of conditions and this makes them one of the most fundamental concurrency abstractions. The big question is: how can the usage of a condition cause a deadlock? When a thread is going to wait for a notification, it blocks, it is added to a wait-set (a set of blocking threads that wait for that condition) and the associated lock is released. Releasing and reacquiring the lock is done behind the screens and luckily is not a task developers need to deal with. But if the waiting thread never receives the notification it is waiting for, the thread won’t wake-up and therefore is in a deadlock.

Imagine there is a fridge with room for beer. If someone wants to grab a beer from the fridge, he opens the door (other people are not able to access the fridge while it is opened) and checks if one is available. If there is, he takes it, closes the door and returns. If there isn’t, he closes the door and waits next to the fridge. When the delivery guy wants to put something in the fridge, he opens the door, places the item, shouts that a new item has been put and closes the door. The guy that is waiting heard the delivery guy, he opens the door, sees that a beer is available, takes it and closes the door.

class Fridge{
	final Object monitor = new Object();
	final List<Beer> itemList = new LinkedList<Beer>();

	void put(Beer beer){
		synchronized(monitor){
			itemList.add(beer);
			monitor.notify();
		}
	}

	Beer take()throws InterruptedException{
		synchronized(monitor){
			while(itemList.isEmpty())
				monitor.wait();
			return itemList.remove(0);
		}
	}
}

The Fridge class is an implementation of the example. It uses a Java monitor, a combination of a condition (wait-set) and its corresponding (intrinsic) lock:

1. the lock is used to synchronize between threads (providing exclusive access)
2. the condition is used to communicate between threads (the waiting and shouting part)

Lets enhance our example and place the fridge in a very small kitchen; a kitchen with room for just a single person. We can prevent multiple people from accessing the small room by adding a door to the kitchen.

class Kitchen{
	final Object monitor = new Object();
	final Fridge fridge = new Fridge();

	void put(Beer beer){
		synchronized(monitor){
			fridge().put(beer);
		}
	}

	Beer take()throws InterruptedException{
		synchronized(monitor){
			return fridge.take();
		}
	}
}

In this Kitchen class, the kitchen-door is implemented by a monitor (only the lock is used b.t.w.). When someone wants to access the fridge he sees that the kitchen-door is open and enters the kitchen, he closes the kitchen-door (he acquires the lock of the kitchen-monitor), he opens the fridge-door (he acquires the lock of the fridge-monitor) and looks inside. When he realizes that the fridge is empty, he waits in the kitchen. When this happens, the lock on the fridge-door is released, but because he stays in the kitchen the kitchen-door remains closed (locked). The consequence is that everybody who want to access the kitchen is going to deadlock:

1. someone that wants to take something from the fridge or wants to refill the fridge is going to deadlock because he can’t enter the kitchen (the kitchen-door is closed) and keeps waiting.
2. the person that is waiting in the kitchen, is going to deadlock because nobody is able to refill the fridge because he has locked out everybody from the kitchen.

This problem is called: ‘nested monitor lockout’. Composing larger structures (e.g. kitchens) from smaller structures (e.g. fridges) is a technique all object oriented developers use on a daily basis (dependency injection, object composition, decorators, template methods, proxies, strategies, etc). But as soon as these structures block and are wrapped within other concurrent structures, you can get in serious liveness problems. So one of the first things I look at is if synchronized structures are wrapped in other synchronized structures and possible do blocking calls (if you are lucky, the methods throw an InterruptedException). But this can be a very challenging task if the concurrent code isn’t localized but scattered all over the place or when the InterruptedExceptions aren’t handled correctly. Maybe in the future techniques like STM (Software Transactional Memory) can help us, but for the moment we need to keep paying extra attention to concurrency control.

For more information about this subject I recommend “Concurrent Programming in Java: Design Principles and Pattern (2nd Edition)” from Doug Lea, Chapter 3.3.4 “Confinement and nested monitors”.

A SERIALIZABLE transaction in classic lock based databases, is implemented by using pessimistic locking. The transaction needs to obtain the exclusive table lock prior to accessing the table. This prevents other SERIALIZABLE transactions and non SERIALIZABLE transactions (these need to obtain the shared table lock) from executing concurrently. The consequence is that a SERIALIZABLE transaction blocks until the exclusive table lock is obtained, or when it is rolled back (eg a timeout, detection of a deadlock, etc).

In Oracle no exclusive table lock is required to implement the SERIALIZABLE isolation level. Instead Oracle is able to take a snapshot of the database and use this snapshot for the duration of the transaction (transaction level read consistency). This is realized by reading the SCN (the System Change Number: a version number that is increased when a transaction commits) when the transaction begins and using this SCN and the data in the rollback segment, to reconstruct previous versions of records. The consequence of this approach is that different transactions could see different snapshots at the same moment (hence Multi Version Concurrency Control). Therefor a SERIALIZABLE transaction is not able to see other updates (so no worries about unrepeatable reads) but also not other inserts/deletes (so no phantom reads).

In short: instead of relying on pessimistic locks to prevent isolation problems, the transaction is able to see the state of the database as it was when the transaction began, even though other transactions could have changed the database in the meanwhile! This is one of the reasons why Oracle is able to perform well under heavy load.

There is one caveat: most transactions do more than just reading (else the READ_ONLY isolation level would have been a better solution). When a transaction modifies (inserts, updates, deletes) records, these records are exclusively locked for the remaining duration of the transaction (these changes are visible inside the transaction of course), independent of the isolation level. But once a transaction ends, these locks are released and without extra protection, it could happen that a SERIALIZABLE transaction updates an older version of the record (a reconstructed one) than the last committed version. This problem is called a lost update.

Luckily Oracle protects us from lost updates with SERIALIZABLE transactions by adding an optimistic lock: when the transaction tries to update a record that has been committed after the transaction began, it knows that a different transaction has modified it and the transaction aborts with the “ORA-08177: can’t serialize access for this transaction”. Just like optimistic locking failures in normal database communication, the standard policy of dealing with this situation is to retry the transaction and hope that the transaction isn’t interfered again.

1. defines what your system can and can’t do
2. a good location for transaction demarcation
3. a good location for security

And although a lot has been written about the service layer (Patterns of Enterprise Applications from Martin Fowler is one of the most famous books about this subject), there are a lot of ways to implement it. In this post I’ll walk through a few examples that are improved step by step.

An example of some bad code I have seen too often:

class BonusController{
	EmployeeDao employeeDao;
	EmployeeService employeeService;

	void someAction(Request request){
		long id = new Long(request.get("id"));
		Employee employee = employeeDao.findById(id);
		employee.setSalary(employee.getSalary()+100);
		employeeService.saveOrUpdate(employee);
	}
}

There is a lot wrong with this code. The most important issues are:

1. the bonus logic is hard to reuse because it is integrated inside the controller, so duplication is not uncommon in such systems.
2. it is hard to maintain/extend because the bonus logic is overshadowed by all the plumbing.
3. it is very easy to bypass any form of validation because the database is exposed by the saveOrUpdate method. If the employeeService is used inside a remoting layer, and you have ‘deserialized’ an employee, god knows what kind of properties are placed in the database when it is saved.

A better, but still bad solution, is the following:

class BonusController{
	EmployeeDao employeeDao;
	EmployeeService employeeService;

	void someAction(Request request){
		long id = new Long(request.get("id"));
		Employee employee = employeeDao.findById(id);
		employeeService.giveBonus(employee);
	}
}

class EmployeeService{
	EmployeeDao employeeDao;

	void giveBonus(Employee employee){
		...execute logic for bonus
		employeeDao.saveOrUpdate(employee);
	}
}

The horrific saveOrUpdate method has been removed and the bonus logic has been moved inside the service. But we are not out of the woods yet:

1. I’m still able to inject an employee that bypasses all checks
2. Transactional behavior is vague at best and erroneous at worst. If multiple transactions are working on the same employee, and if there is no form of offline locking, then one transaction is able to overwrite the changes of the other: this problem is called the lost update problem. The consequence of a lost update is that we could loose a bonus! Beside the lost update there are other problems: Imagine what happens when one transaction has deleted the employee and the second one commits. If all logic is executed inside a single transaction, you can make use of standard solutions to prevent these problems (optimistic/pessimistic-locks, isolation levels etc).

Another issue I find smelly, is that no explicit transaction is used to access the employeeDao inside the controller. Most databases are friendly enough to create a transaction if one is missing, and in the previous example it is quite clear that an employeeDao is used inside a controller, but I have seen my share of deeply tucked away dao’s inside presentationlayer support objects like Spring editors. I think this is a good example of misuse of technology.

The simplest solution to the problems above is the following:

class BonusController{
	void someAction(Request request){
		long id = new Long(request.get("id"));
		employeeService.giveBonus(id);
	}
}

class EmployeeService{
	void giveBonus(long employeeId){
		Employee employee = employeeDao.findById(employeeId);
		....bonus logic
	}
}

All previous mentioned problems are solved, and even the controller has been simplified. This approach even is agnostic to the technique you use to realize the bonus logic. You could create a transaction script:

class EmployeeService{
	void giveBonus(long employeeId){
		Employee employee = employeeDao.findById(employeeId);
		employee.setSalary(employee.getSalary()+100);
	}
}

You could place it inside the domain object of you have a richer domain model (something I’m for).

class EmployeeService{
	EmployeeDao employeeDao;

	void giveBonus(long employeeId){
		Employee employee = employeeDao.findById(employeeId);
		employee.giveBonus();
	}
}

And if your bonus logic is very complex, you could even decide to use a domain service (see Domain Driven Design from Eric Evans, chapter 5, subject: Services for more information) that contains the actual logic:

class EmployeeService{
	EmployeeDao employeeDao;
	BonusDomainService bonusDomainService;

	void giveBonus(long employeeId){
		Employee employee = employeeDao.findById(employeeId);
		bonusDomainService.giveBonus(employee);
	}
}

For those that have watched carefully, the EmployeeService provides another very important function: it provides a layer of abstraction. The outside world is not coupled to the implementation, and this means that changes in the implementation won’t effect the clients (presentationlayer, remoting layers).

Visibility problems

Most programmers don’t have much experience with multi threading and have a very simplistic view on reality: if a thread makes a change to a variable, this change is directly visible to all other threads that access the same variable. This view is called sequential consistency, but the problem is that no virtual machine automatically provides this view.

The reason for this apparently faulty behavior is performance. Most variables are not shared between threads, so no special instructions (memory barriers) have to be added to make the program sequentially consistent. If these instructions were automatically added to all variable accesses, it would greatly reduce performance because a lot of optimizations would be excluded from being used. To name a few:

1. usage of super fast local-memory (like caches and cpu-registers). It also could reduce the performance benefits of multi-core systems because the increased access to main memory would become the bottleneck.
2. reordering of instructions to increase the chance of a cache hit.

The consequence of a visibility problem, is that a value written to a variable by one thread, doesn’t have to be visible in other threads (maybe never). This could lead to an easy to detect NullPointerException, but it can also lead to much harder to detect problems like shown in the following example:

public class SomeRunnable implements Runnable{
    private boolean stop = false;

    public void stop(){
        stop = true;
    }

    public void run(){
        while(!stop){
                System.out.println("hello");
        }
    }
}

There are a few reasons why this runnable could fail to stop:

1. stop is not safely published: the thread that executed the run method, doesn’t need to see a change in the stop value. A possible scenario would be: thread1 (the thread that executed run) is running on cpu1 (with cache1). Thread2 (the thread that calls stop) is running on cpu2 (with cache2). When thread1 runs, it needs the value of stop, sees the initial false value and places it in his cache. When thread2 executes stop, stop is set to true. But it might happen that this value remains in cache2 for an undetermined time and isn’t flushed to main memory, so thread1 never sees the new value when it happens to read from main memory. Even if the value is flushed to main memory, thread1 could still be reading the stale value from cache1.
2. because the access to the stop variable isn’t safely published, and the value doesn’t change in the loop, the ‘compiler’ (JIT, cpu etc) could decide to replace the variable read by the constant ‘true’ in that loop.

Visibility problems can lead to hard to detect errors and unpredictable behavior, and you definitely want to keep them out of your system. And although most cpu’s are not subject to visibility problems (because of strong Memory Coherence), betting on this behavior is asking for problems (the application also wouldn’t be platform independent).

Spring applications and visibility problems

I expect that a lot of Spring applications are subject to visibility problems. If you look at singleton beans for example (dao’s, controllers, managers etc), these beans often are shared by many threads, to name a few:

• threads from the servlet container
• threads from remoting middleware
• threads from jmx
• threads from triggers or other internal threads

Take a look at the following example:

public class EmployeeManagerImpl implements EmployeeManager{

    private EmployeeDao employeeDao;

    public setEmployeeDao(EmployeeDao employeeDao){
        this.employeeDao = employeeDao;
    }

    public void fire(long employeeId){
        Employee employee = employeeDao.load(employeeId);
        employee.fire();
    }
}

And the bean configuration that belongs to it:

    <bean id="employeeManager" class="EmployeeManagerImpl">
        <property name="employeeDao" ref="employeeDao"/>
    </bean>

The visibility problem here, is that employeeDao, set by the thread that constructed the employeeManager, isn’t safely published and doesn’t have to be visible in other threads. If one of those threads calls the fire method on the EmployeeManagerImpl, it could lead to a NullPointerException because that thread still sees a null value for the employeeDao.

The problem can be solved in a few ways:

1. make employeeDao volatile. Although it sounds quite strange, because the employeeDao is not going to change after it has been set, volatile variables are always safely published. Personally I don’t like this solution much because it is misleading (the value is never going to change after the object has been constructed) and it also reduces performance (the variable always has to be read from main memory instead of local memory).
2. make employeeDao final, because final variables also are safely published. The problem with final variables is that they only can be set by constructor based Dependency Injection (DI) and Spring promotes setter based DI. Personally I prefer constructor based DI, because it is a lot easier to guarantee class invariants. Although those setters only are visible in an implementing class, and not in the interface, I still don’t feel happy about because it also makes classes harder to understand. But constructor based DI isn’t perfect either; I guess most of us have struggled with large constructors.

Problem not that bad?

Before going into detail why the problems maybe are not that bad, I need to explain the new Memory Model found in Java 5 and higher (JSR-133) (see footnote). The model describes under what kinds of conditions a variable will be visible in other threads and it also shields you from reasoning about caches (they are not mentioned in the Java Memory Model).

The model is expressed in terms of actions:

1. read/writes to normal/volatile/final variables
2. lock acquire/release
3. thread start/join

The model also contains a set of happens-before rules between actions. If a happens-before rule applies between action1 and action2, then all changes made by action1 are visible in action2. Examples of happens-before rules are:

1. Program Order rule: each action in a thread happens-before every action in that thread that comes later in the program order. It is important to realize that normal variable reads/writes, can be reordered as long as they keep the ‘within-thread as-if-serial semantics’: the reordering should not be visible inside the thread.
2. Volatile variable rule: a write to a volatile variable happens-before every subsequent read of that same variable (that is why making the employeeDao volatile works).
3. Monitor lock rule: an unlock on a monitor lock happens-before every subsequent lock on that same monitor lock. (The same goes for the the new Lock found in Java 5.
4. Transitivity rule: if action1 happens-before action2, and action2 happens-before action3, then action1 happens-before action3.

If there is a happens before relation between the write and the read of a variable, then it is safely published.

Safe handoff

These happens-before rules can be used to check if there is a happens-before relation between two actions. Take a look at the following example:

int a=0;
volatile boolean b=0;

void initialize(){
        a=console.readInteger();
        b=console.readBoolean();
}

void print(){
        print(b);
        print(a);
}

A possible ordering of actions could be:

Because action1 happens-before action2 (program order) and because action2 happens-before action3 (volatile variable rule) and because action3 happens before action4 (program order), action1 happens before action4 (remember that the happens before rules are transitive). This means that the change in action1, is visible in action4. This technique is called ‘safe handoff’ (aka ‘piggybacking on synchronization’).
Safe handoff uses the safe publication of a variable X (in this case b), to safely publish all unsafely-published-variables that are changed before X (in this case a).

Safe handoff in Spring

The Spring applicationcontext also provides safe handoff (only in Java 5 and higher): after a singleton bean is created, it is placed in a singletonmap. When it is needed, it is retrieved from that map. But instead of using the ‘volatile variable’ rule, it uses the ‘monitor lock’ rule (action4 and action5)

The following table shows a very simplified ordering of actions (variable mapentry.ref is the variable ‘X’ from the ‘save handoff’ technique):

This means that there is a happens before relation between action2 and action8, so the value set in the employeeDao (action2), is visible when it is read by another thread (action8). That is why standard Spring singletons don’t have visibility problems. If you want to see it for yourself, check out the DefaultSingletonBeanRegistry and the ‘public Object getSingleton(String beanName, ObjectFactory singletonFactory)’ method

Conclusion

I’m glad that standard singleton beans aren’t subject to visibility problems (under Java 5 and higher), but I don’t think it is the correct way to deal with concurrency control in Spring (or in general):

1. objects can’t be safely used in a different environment. So your objects are tied to the (current version of the) Spring framework to make them work correct.
2. the save handoff behavior is not guaranteed by Spring, and it only works under Java 5 and higher. The behavior is undefined in older JVM’s.
3. are beans with different scopes than singletons free from visibility problems?

I don’t want to depict Spring as a bad product: it really is amazing and I’m always happy when I can use it on a project. But I do think that developers should be more careful when writing code that is used in a multi threaded environment.

A part of this responsibility should be taken bij developers themselves, if you write threadsafe code, you need to know what you are doing. But I also think the guys behind Spring should take part of this responsibility: not all developers have a lot of experience with writing multi-threaded code and I guess most don’t know anything about the Java Memory Model. If these issues are not mentioned in the Spring documentation, a lot of developers will remain agnostic and this is going to lead to problems in the future.

Footnote:
If you want a more in depth explanation of the new Java Memory Model, I suggest ‘Java Concurrency in Practice’ from Brian Goetz, Tim Peierls, Joshua Bloch, Joseph Bowbeer, David Holmes and Doug Lea, or check out the following website The Java Memory Model

I want to thank colleagues, acquaintances and the guys from the concurrency mailing list for proofreading this post.