Its a very common programming technique to create any number of URLClassLoader instances repeatedly in order to load the new implementation of the classes/resources from the same location with the help of new class loader instances.
In fact, some well-known tools like plexus-compiler jar makes heavy usage of IsolatedClassLoader to implement the above concept.
Lets focus on the actual problem.
We have a large maven project which has 200 jars in maven repo.
Any code change in source file fires – MavenBuilder …..
– which invokes JavacCompiler (plexus-compiler jar)
– in turn creates IsolatedClassLoader cl = new IsolatedClassLoader() [extends URLClassloader] ..
– adds the list of 200 jars in urlClassPath – cl.addURL(URL)
– loads javac.Main along with all jars in classpath and reflectively fires compilation.
So for N save-operations, N instances of IsolatedClassloaders are created.
In principle, once the application clears all references to a loader object, the garbage collector and finalization mechanisms will eventually ensure that all resources (such as the JarFile objects) are released and closed.
But in reality, the application goes OutOfMemory !
We used optimal concurrent gc strategy – Xgcpolicy:optavgpause.
But still the app running out of memory very quickly.
Then Heap dump analysis showed there are some 100 instances (Uuh !) of isolatedClassloaders each of which is holding onto unclaimed ZipFileIndexEntries(.. guess what .. indexEntry for all the jars loaded on each instance of class laoder..)
Well ! looks like.. since a new URL Class Loader is created before closing the resources in previous loader, GC is confused and does not reclaim the previous class loader ! This causes problems for applications which need to be able to GC in a predictable and timely fashion. It is a particular problem on Windows, because open files cannot be deleted or replaced.
So are we missing something trivial here ? Is plexus-jar doing something wrong ?
A BIG Emphatic YES !
We should always use sun.misc.ClassLoaderUtil to release an url class loader and file resources held therein.
Lets now fix the issue.
IsolatedClassLoader loader= new IsolatedClassLoader();
c = loader.loadClass( “com.sun.tools.javac.Main” );
ok = (Integer) compile.invoke(args)
//Now that you are done with compilation, get rid of loader ..
*** loader.close () Or sun.misc.ClassLoaderUtil.releaseLoader(loader)
////// -> allows graceful release of loader and all resources
*** Thankfully JDK 7 URlClassloader has implemented close() method to give the caller a chance to invalidate the loader, so that no new classes can be loaded from it. It also closes any JAR files that were opened by the loader. This allows the application to properly delete or replace these files and, gracefully create new loaders using new implementations.
The bottom line is wherever we create seemingly harmless URlCLassloader in a repeated manner (say to generate / compile code) holding onto a good many jars in url classpath; we should always close the loader instance.