So we decided to search for the cause of the memory leak. Soon we found out, that there were some classes from the web application which the classloader could not remove from the PermGen space because they were "linked" to classes which were loaded by the standard classloader. There could be several causes for that and using the right tools (jmap and jhat from the JDK) plus some small programm to find reference chains we found the culprits:
* MySQL Connector/J (see http://bugs.mysql.com/bug.php?id=36565)
* iBatis (see https://issues.apache.org/jira/browse/IBATIS-540)
* Tomcat (see https://issues.apache.org/bugzilla/show_bug.cgi?id=46221)
* and some we could fix by just moving some JARs to the right places (see also here and here).
Identifying the subjects was an iterative task - fixing one leak caused appearing the next one ... We did not know that there were so many candidates at the beginning. We could fix iBatis by switching to a newer version, MySQL, JabRef and Tomcat were a little harder to fix.
For JabRef we had to modify the source code such that it does not start AWT. Additionally, a Tomcat LifecycleListener kills the java.util.prefs.FileSystemPreferences after webapp shutdown using awful Java introspection hacks:
final Class clazz = CleanupListener.class.getClassLoader().loadClass("java.util.prefs.FileSystemPreferences");
final Field f = clazz.getDeclaredField("syncTimer");
final Timer timer = (Timer) f.get(null);
To fix the MySQL bug, the listener ensures on the startup of the web application that the MySQL connection class is loaded before the web app and by the standard classloader, such that the cancellation timer threads (which is the cause of the leak) don't block unloading of the webapp. The loggers from the StandardContext in Tomcat (which are loaded via the webapps classloader - for whatever reason) are also killed by the listener.
After several weeks of work we have a leak free application. The bad thing is that every library we are using can bring back a leak and if we are not careful the leak will be back quicker as we like. Unfortunately we are not aware of a method which we could put into the Tomcat or into our application which just checks for memory leaks.
Hope you found this interesting and good luck with your own applications ...