Tuning and optimizing a JVM garbage collector can be a very rewarding process in terms of meeting project specific throughput and response time requirements. The OpenJDK/Oracle JVM becomes better with every release but the base settings are often a bit conservative. My baseline set up for an application server’s garbage collector using the Sun/Oracle JVM is the following:
export JAVA_OPTS="-Xms1024m -Xmx1024m -XX:MaxPermSize=256m -XX:MaxNewSize=448m -XX:NewSize=448m -XX:SurvivorRatio=6 -XX:+UseConcMarkSweepGC"
I found that this configuration works quite well out of the box for JBoss 7.1.x and GlassFish 3. It provides enough memory for typical Java EE applications, keeps garbage collections times low and garbage collections pauses are short and happen at a higher frequency. Load tests using JMeter show a very consistent heap usage and no big spikes and pauses like basic configurations.
I don’t want to go too deep into the topic because there’s lots and lots of literature on tuning the JVM and JVMs are changing too. Note however, that increasing the heap far beyond 2GB may impact desired qualities on Oracle JVMs even in 64 bit machines. Use the VisualGC tool to monitor garbage collector performance.
Think about running multiple JBoss or GlassFish instances on the same machine in their on environment. Using domain-driven design get a clean structure, message-based integration (instead of a shared database) and sticking to highly cohesive applications (Entity, Control, Boundary) usually presents lots of opportunities to decouple applications.
If you really need vast amounts of heap give other JVMs a try. AZUL Systems hast a very interesting JVM called Zing and claims to have 40x better response time than a tunes Oracle JVM. According to the company their JVM usually runs with 100 to 200GB of heap. If your servers run with 144GB RAM configurations this may be a good investment.