After some changes recently, my application began to hang. I run the Glassfish V2 application server and so went for a stacktrace to ascertain the responsible Java code:
asadmin generate-jvm-report --type=thread
This showed that the server was waiting on the database to return from an update query. This rang alarm bells - a locking wait I suspect...
I decided to check with Postgres which query was waiting:
SELECT procpid, current_query FROM pg_stat_activity;
This showed me that one particular update to my c_account table was waiting.
I was using two transactions: 1 & 2. The code which started tx1 then starts tx2 before committing tx1.
Transaction 1:
Inserts a logging record, referencing c_account with a foreign key.
Transaction 2:
Changes the account bar flag on c_account with an SQL update.
This scenario caused a deadlock, since tx1 takes a row-exclusive lock on the log table and any rows referenced by its foreign keys. When I go to modify the flag on c_account, then tx2 has to wait until tx1 has completed and of course, this will never happen.
Two methods to get around this:
1) Better transaction demarcation. Carefully avoid insert/update referencing the same entity.
2) More locking-aware table design. Have a one-one table for flags such that new records may reference the core entity, without locking the one-one tuple.
For now I am opting for option 1, but will keep option 2 in mind for future table design.
Wednesday, 28 July 2010
Subscribe to:
Post Comments (Atom)
In the end decided to modify the database since I didn't like having to limit tx1's scope and remit because of limitations of the data model.
ReplyDelete