Hibernate and Multi-Threading

Maarten Winkels

When you use Hibernate for ORM and come across some functionality that requires multi threading, there are many pitfalls that might make life difficult for you. This blog will focus on those problems. Conclusion is: don't use hibernate managed objects in multiple threads.

Let's look at a very simple domain: clients with payments. Each client has a set of payments

@Entity public class Client {
	...
	@OneToMany(cascade=ALL)
	private Set payments = new HashSet();
	
	public void addPayment(double amount) {
		payments.add(new Payment(amount));
	}
	
	public double getTotal() {
		int total = 0;
		for (Payment payment : payments) {
			total += payment.getAmount();
		}
		return total;
	}
	
}

@Entity public class Payment {
	...
	public Payment(double amount) {
		this.amount = amount;
	}
	...
}

Now a very simple test is to save create a client with some payments, then reload it and test the total number of the payments again.

	public void testSimpleClientWithPayment() throws Exception {
		Client client = new Client("me");
		client.addPayment(12.0);
		client.addPayment(5.0);
		
		assertEquals(17.0, client.getTotal());
		
		Serializable id = dao.save(client);
		
		dao.flushAndClear();
		
		client = dao.get(id);
		
		assertEquals(17.0, client.getTotal());
	}

Of course this all works like a charm and there are no problems. The project drifts easy along its ever lasting path and you keep on coding.

Introduce threads: Lazy Loading problems

At some point one of the wise-ass architects comes up with a scheme where reports of the totals of all clients have to be printed on a regular basis in separate threads. The reports are sent by mail or some other lame excuse for introducing this fancy new threading thing. So to keep him happy you create a nice service that can do that.

public class ReportingService {

	private final class Reporter implements Runnable {
		private final Client client;

		private Reporter(Client client) {
			this.client = client;
		}

		public void run() {
			System.out.println(client + ": " + client.getTotal());
		}
	}

	public void reportTotals() {
		for (final Client client : dao.findAllClients()) {
			executorService.submit(new Reporter(client));
		}
	}
	...
}

Of course you test the service with a number of payments, and everything works out fine... until the feature is brought into production and the logs start filling up with these exceptions:

21:21:11,562 ERROR [LazyInitializationException, LazyInitializationException.] failed to lazily initialize a collection of role: domain.Client.payments, no session or session was closed
org.hibernate.LazyInitializationException: failed to lazily initialize a collection of role: domain.Client.payments, no session or session was closed
	at org.hibernate.collection.AbstractPersistentCollection.throwLazyInitializationException(AbstractPersistentCollection.java:358)
	at org.hibernate.collection.AbstractPersistentCollection.throwLazyInitializationExceptionIfNotConnected(AbstractPersistentCollection.java:350)
	at org.hibernate.collection.AbstractPersistentCollection.initialize(AbstractPersistentCollection.java:343)
	at org.hibernate.collection.AbstractPersistentCollection.read(AbstractPersistentCollection.java:86)
	at org.hibernate.collection.PersistentSet.iterator(PersistentSet.java:163)
	at domain.Client.getTotal(Client.java:43)
	at service.ReportingService$Reporter.run(ReportingService.java:18)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:417)
	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:269)
	at java.util.concurrent.FutureTask.run(FutureTask.java:123)
	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
	at java.lang.Thread.run(Thread.java:595)

But you tested that, right? What could possibly be wrong? Well, the collection of payments from a few of the clients are not yet initialized (=loaded from the database) when the client is handed to the executor for reporting. This is no problem: Hibernate can lazily load the collection whenever this is needed. For that purpose, it replaces the collection with a special implementation, that keeps a reference to the Session that was used to load the containing entity (the client). When we call payments.iterator(), the collection is loaded from the database through the Session. As long as this Session is open, the code will work.

Sessions, however, are going to be closed at some point. Using springs transaction management, for example, they are often associated with a transaction and closed after completion of that transaction. Now if the transaction that loads a client with a lot of payments shoots of all these tasks and then completes before the tasks are executed, the session will close and the tasks are left without the option to lazily load the payments. Hence the lazy loading problems. In a test environment it is often hard to simulate this, because there is less data and your testing infrastructure might keep transactions and sessions open until everything is executed. You will mostly get burned in production.

So, the solution is simple, right? Just load the payments before you execute the task:

		...
		for (final Client client : dao.findAll()) {
			// --> Solution:
			Hibernate.initialize(client.getPayments());
			executorService.submit(new Reporter(client));
		}
		...

That seems simple enough. You can also initialize the set by executing a method, for example .size(), or by fetching it eagerly when loading the client.
So, no more stacktraces in the production log, you saved the company!

Missing Updates

Now the business analysts have come up with a new idea: The printing of the totals is too simple, the clients want a record in the database of the total per client every week and they want to know when the last total was generated for each client. So you extend your ReportingService with a new method and a new task:

	...
	public class ReportGenerator implements Runnable {

		private final Client client;

		public ReportGenerator(Client client) {
			this.client = client;
		}

		public void run() {
			dao.save(new Report(client, client.getTotal()));
			client.setLastCreated(new Date());
		}

	}
	...

Now if you execute this task in your test environment, you notice something strange: The reports are added to the database, but the "lastCreated" field is not updated. What can be the problem? Hibernate should perform transactional write-behind with dirty checking: When the transaction is completing, it should check that the client object has changed and persist the changed values. This works normally, why is this different?

The dirty checking is only performed in the session in which an object was loaded. The client object was loaded in a different session, on which the transaction has already been completed. When the report is created, this is saved in a separate session. The infrastructure of the dao takes care associating the session with the current thread and of instantiating a new session when necessary.

Locking with no mode

So how do you solve this? To be able to do updates on an object loaded from a different session, you'll have to reconnect it to a new open session. The .lock(.., LockMode.NONE) operation can be used for that. You have to make sure that the session remains open until the updates are done. To ensure this, we use a new transaction for the new report action:

		...
		public void run() {
			new TransactionTemplate(transactionManager)
					.execute(new TransactionCallbackWithoutResult() {
						@Override
						public void doInTransactionWithoutResult(TransactionStatus status) {
							dao.lock(client, NONE);
							dao.save(new Report(client, client.getTotal()));
							client.setLastCreated(new Date());
						}
					});
		}
		...

Using this solution in combination with the previous solution, however, will result in new exceptions:

Caused by: org.hibernate.HibernateException: Illegal attempt to associate a collection with two open sessions
	at org.hibernate.collection.AbstractPersistentCollection.setCurrentSession(AbstractPersistentCollection.java:410)
	at org.hibernate.event.def.OnLockVisitor.processCollection(OnLockVisitor.java:38)
	at org.hibernate.event.def.AbstractVisitor.processValue(AbstractVisitor.java:101)
	at org.hibernate.event.def.AbstractVisitor.processValue(AbstractVisitor.java:61)
	at org.hibernate.event.def.AbstractVisitor.processEntityPropertyValues(AbstractVisitor.java:55)
	at org.hibernate.event.def.AbstractVisitor.process(AbstractVisitor.java:123)
	at org.hibernate.event.def.AbstractReassociateEventListener.reassociate(AbstractReassociateEventListener.java:79)
	at org.hibernate.event.def.DefaultLockEventListener.onLock(DefaultLockEventListener.java:59)
	at org.hibernate.impl.SessionImpl.fireLock(SessionImpl.java:584)
	at org.hibernate.impl.SessionImpl.lock(SessionImpl.java:576)
	...

This is because we are attempting to connect the collection, that we so cunningly initialized before, to the new session as well. This collection however, will attempt to keep a reference to one open session, but not two.

On the bright side, we do not have to initialize the session anymore if we connect the client to a new session: lazy loading will work again. In some situations however, it is very hard to determine which objects are already initialized and which aren't and when to connect which objects to the new session and so on... Using the lock operation is often a tedious job!

Just pass the ID... silly!

After battling the problem for a while, you give up and settle for a much easier scheme: You store the object in the database and pass the ID-value to the worker thread. That thread can now reload the data from the database and safely work on it.

Conclusion

Passing Hibernate managed objects (like collections or proxies) to other threads might result in some hard to trace problems, like:

  • Lazy Initialization problems
  • Missing Updates
  • Locking exceptions

It is therefore often better not to pass Hibernate managed objects to other threads, but to save the objects and reload them by ID from the database in the other tread.

Comments (15)

  1. Nirav Thaker - Reply

    February 8, 2009 at 6:25 pm

    Except for Lazy initialization problems I don't see why hibernate managed objects in particular has multithreading issues (note that this is problem with the way you handle transactions).

    Even if you wrote pure SQL, you will have these legitimate concurrency issues. As developers we are expected to be aware of such problems.

    It is hilarious to see 'updates' for reporting, go figure.

  2. Jens Schauder - Reply

    February 8, 2009 at 9:09 pm

    While these problems are ugly enough to be center of a blog article, they have nothing to do with multithreading. They all can be reproduced within a single thread. (I know because I met them all in person in a swing project which just used the EDT 😉

  3. Maarten Winkels - Reply

    February 9, 2009 at 8:37 am

    What makes hibernate especially ugly with respect to multi threading is that the Hibernate managed objects (hibernate implementations of collections and proxies) keep a reference to the session in which they were loaded. They'll try to do any updates through that session. If you pass the object to another thread, but do transaction handling on a per thread basis (which is the defacto standard with Hibernate) you'll run into a lot of problems.

    You can run into the same kind of problems in a single thread if you don't handle your transactions correctly, but with multi threading they are all the more likely, even in otherwise 'simple' scenarios.

    Regards.

  4. Peter Veentjer - Reply

    February 10, 2009 at 11:41 am

    Hi Maarten,

    nice to see some stuff about concurrency control and databases. Using the id is a very good solution imho because it provides very clear transactional behavior. In some cases you need to make sure that the entity which id is passed, hasn't changed in the mean while. In those cases you could introduce a versionedid (an id in combination with the version). This uniquely identifies an object over time.

    A small remark about this example client: I would not model the Payments as part of the client (not all relations have to be modelled as a java reference) but use a dao/repository call instead. A client can live without its payments. This simplifies the domain as well imho.

  5. Francesco - Reply

    November 24, 2009 at 9:57 am

    Very interesting topic.
    But what about the "main" thread?
    Imagine you do some modification to an entity in the worker thread which you would like to see reflected in the main thread (for example a list that threads the modification on its items).
    How to refresh the entity loaded in the original session?
    Reload? Refresh? Merge? All these throw different exceptions...

    Seems that all the time you save with basic CRUD operation is payed back by these kind of tricky situations...

  6. Diaa Kasem - Reply

    December 19, 2009 at 11:22 pm

    Wonderful, I really needed these tips.
    Thank you.

  7. jens - Reply

    February 10, 2010 at 3:55 am

    Hello,

    absolutely greate and helpfull article!!!! I just ran into these problems myself and it was very interesting helpfull to read your article. Thanks a million times, now I know how to solve the problem.

    thanks jens

  8. Raghavendra - Reply

    April 13, 2010 at 9:02 pm

    Hi

    It is very good .How better way we can use threadpoolexecutor for this hibernate intialize process

    • Helen - Reply

      October 1, 2015 at 10:53 pm

      Finding this post has anreeswd my prayers

  9. Ramki - Reply

    May 9, 2011 at 5:31 pm

    Hi,
    I have the similar problem in my project. We have one jms listener to listen one queue. when request comes to that queue, onMessage() will take that request and delegate that request to ExecutorService(java 1.6) which in turn call one controller to execute that task(ExampleCotroller). in side that controller i am loading parent object(which has some child entries). while getting the child entries we are getting lazy initialization exception, saying session closed or no session.

    your help is appreciated here....

    Thanks,
    Ramki.

  10. Dave Ritz - Reply

    May 20, 2011 at 9:03 pm

    Just wanted to say thank you very much.
    We had a similar problem and solved it by using the ID.

    Dave

  11. [...] of all the problems we can meet when trying to use Hibernate in a multithreaded application (1st clue, 2nd clue, 3rd clue, etc.), I was thinking of another solution: implementing the logical part [...]

  12. Mohamed - Reply

    August 22, 2012 at 5:35 pm

    Thanks so much for this article. It helps me to implement a fix of a stressed situation.

    Yes Hibernate has some internal voice that cannot be understood by human people 🙂

  13. Shane - Reply

    July 23, 2013 at 12:09 am

    Maarten, you have saved my life. Thank you.

  14. Pierrick - Reply

    January 13, 2016 at 5:33 pm

    Thanks so much for this article ! Very usefull

Add a Comment