Analyzing Heap Dumps (Hibernate Session Objects Create Once Only)

Shallow size of an object is the amount of memory allocated to store the object itself, not taking into account the referenced objects. Shallow size of a regular (non-array) object depends on the number and types of its fields. Shallow size of an array depends on the array length and the type of its elements (objects, primitive types). Shallow size of a set of objects represents the sum of shallow sizes of all objects in the set.

Retained size of an object is its shallow size plus the shallow sizes of the objects that are accessible, directly or indirectly, only from this object. In other words, the retained size represents the amount of memory that will be freed by the garbage collector when this object is collected.

Several memory analysis tools will be used:

VisualVM - bundled with the JDK and useful for realtime analysis and produce heap dumps remotely

JConsole - bundled with the JDK and useful for realtime and over-time analysis Eclipse Memory Analyzer Tool ("MAT") - excellent feature for analyzing heap dumps and producing reports

Execute jhat on the individual heap dumps

The jhat tool -- distributed as part of the JDK from version 6 onwards -- can provide another avenue of investigation by parsing the heap dump and providing a web server for one to browse and inspect memory statistics and contents.

Heap Analysis Example

Upon analyzing the heap dumps -- starting with example.hprof -- and running the "Leak suspects report", or by just looking at the size of one particular system classloader (of type sun.misc.Launcher$ExtClassLoader), we can quickly see if any object is having more memory...

In our case org.hibernate.impl.SessionFactoryObjectFactory is loading more memory. Its INSTANCES field (of type org.hibernate.util.FastHashMap) was holding ~893 HashMap.Entry instances (just Node types as it is a linked list implementation), consuming the majority of the aforementioned memory.

Looking at the largest (which also happens to be the first) HashMap.Entry in the INSTANCES Map whose size is ~529KB, we can see its queryPlanCache field holds ~33KB of memory, easily the largest of its fields (of which there are 21 excluding the class definition). Within queryPlanCache of type org.hibernate.util.SoftLimitMRUCache its planCache fields holds java.lang.String instances of the queries it is responsible for running.

Looking through the queryPlanCache field of each SessionFactoryImpl we can see many instances of these XX queries being cache, all with individual strings representing different values for the query (i.e. each query is unique).

To summarise: there is one instance of org.hibernate.impl.SessionFactoryObjectFactory using ~561MB of memory, which has an INSTANCES field of type org.hibernate.util.FastHashMap, which holds a HashMap with ~893 elements. Each of these elements is a org.hibernate.impl.SessionFactoryImpl holding cached SQL queries, most -- if not all -- of which are talking to XX Table.

The cause is creation of session factories on the stack (i.e. local scope). Invoking sessionFactory.buildSessionFactory() which calls the SessionFactoryImpl constructor which calls SessionFactoryObjectFactory.addInstance(uuid, name, this, properties);. This addInstance method adds each SessionFactoryImpl instance (this in the preceding .addInstance(...) example) to a static map called INSTANCES.

Upon moving the session factory to a static final field and thus only calling buildSessionFactory() once during class static initializer invocation.