Error: "java.lang.OutOfMemoryError"

  • 3173888
  • 10-Oct-2007
  • 27-Apr-2012

Environment

Novell Portal Services 1.5
Novell exteNd Director 4.1 Standard Edition
Novell iManager 2.01
Novell eGuide 2.1.x
NetWare 6.0
OES on NetWare 6.5
Tomcat 3.3
Tomcat 4.1.18
Apache 1.3.33 for NetWare
Apache 2.0.52

Situation

Error: "java.lang.OutOfMemoryError"
Getting out of memory errors when using Novell Portal Services 1.5, exteNd Director 4.1 Standard, or iManager 2.01 on NetWare 6.0.
Portal starts running slow and does not allow new users to log in when exceeding approximately 480xxxxxx of Total Committed JVM Memory
Restarting Tomcat temporarily resolves the problem (until memory reaches capacity again)

Resolution

Verify that the server is running the latest support pack
Procedure

(1) Depending on which version of Tomcat you are running, open your TOMCAT33.NCF or TOMCAT4.NCF file found in the SYS:/TOMCAT/33/BIN or SYS:/TOMCAT/4/BIN directory. Here you will need to add two parameters to the file. One for the minimum heap size and one for the max heap size. The parameters are as follows:

-Xms128m -Xmx256m

(2) Insert these parameters into the TOMCAT33.NCF or TOMCAT4.NCF file. At the bottom of the file you will find a line similar to the following:

java -envCWD=$TOMCAT_HOME -classpath $TOMCAT_CLASSPATH -Dtomcat.home=SYS:\tomcat\33 org.apache.tomcat.startup.Main -f sys:/tomcat/33/conf/nwserver.xml %1

(3) Add the heap size parameters to the this line, right after the"java" directive. The new line will look as follows.

java -Xms128m -Xmx256m -envCWD=$TOMCAT_HOME -classpath $TOMCAT_CLASSPATH -Dtomcat.home=SYS:\tomcat\33 org.apache.tomcat.startup.Main -f sys:/tomcat/33/conf/nwserver.xml %1

** IMPORTANT ** Each java process is allocated 512 meg of user space within the RAM that it can use. Some of that is used for java threads, socket communication, etc. So what is really available is not much more than 400 meg which can be allocated to a java process. This information can be seen at the server by typing in the command JAVA -SHOW and then finding out what the ID is for the Tomcat java process. Then at the server console you can type JAVA -SHOWMEMORY# with the # representing the Tomcat process ID. Then switch over to the logger screen and you should see how much memory is being allocated within the Tomcat process. An example is shown below:

JAVA -SHOW

Classname ID
========================================== =========
org.apache.tomcat.startup.main..................................... 390

JAVA -SHOWMEMORY390

Memory Statistics For Class: org.apache.tomcat.startup.Main
-------------------------------------------------------------------------------------------
Reserved Heap: 68161536
Committed Heap: 11124732
Reserved Virtual Memory Pool: 67108864
Committed Virtual Memory Pool: 11272192
NLM Data Memory: 225280
Per Thread Data And OS Stacks: 3403776
Virtual Memory Pool Overflow: 0
JVM Tracking Memory: 23381
Socket Communication Memory: 101616
-------------------------------------------------------------------------------------------
Total Committed Virtual Memory: 22622204
Total Physical Memory: 3528773
Total Committed JVM Memory: 26150977

** NOTE ** The main statistics to look at here are the reserved heap and the committed heap. The reserved heap represents our -Xmx switch. This is the maximum memory that is allocated to the individual java process. What we show here by default is 68161536 which equates to 65 meg. The committed heap represents the -Xms switch. This is the minimum amount of memory that will be committed to the individual java process. In this instance we have 11124732 which equated to 8 meg of committed memory.

(4) To verify that our switches have changed our committed and reserved heap sizes we will need to take down java and then restart tomcat. See example below:

JAVA -EXIT

** IMPORTANT ** If the console comes back with MODULE JAVA.NLM UNLOADED, then you can startup tomcat by typing in TOMCAT33 or TOMCAT4 at the console prompt. If a message is immediately returning indicating "cleaning up JAVA resources in the background", and you have a console prompt, then you can type in JAVA -EXIT again to force java down.

(5) Once tomcat is started again you can check the memory again and it should now show up.

Memory Statistics For Class: org.apache.tomcat.startup.Main
-----------------------------------------------------------------------------------------------
Reserved Heap: 272633856
Committed Heap: 136314876

** TROUBLESHOOTING ** If you do not show the new memory size then one of the following probably happened.
(A) You reloaded Tomcat before JAVA was completely unloaded.
(B) You specified a maximum heap (-Xmx) that was too large. The most that this can be is around -Xmx386m without addition configuration.
(C) Your -Xms parameter is larger than your -Xmx parameter..
If this does not resolve the memory issues for your application, proceed with the following procedure:

On NetWare 6 we have the option of loading the AUTOEXEC.BAT file with a –u switch to increase the amount of memory that can be used by the JVM. This switch can be used in conjunction with java heap parameters to utilize and dedicate more memory to memory intensive Web Applications. Examples of memory intensive web application on NetWare 6 are Novell Portal Services 1.5, exteNd Director 4.1 Standard, eGuide 2.1.x, iManager 2.0x There are a few things that should be noted before using the –u parameter. The –u parameter allows you to specify a higher amount of memory than exists on your server. This could create problems if you are trying to specify memory that you don’t have. So what should be done to avoid this? Well, don’t exceed the amount of RAM that you have on your server with the –u parameter. In fact it would be well advised for you to not go above three quarters of the amount of RAM that you have. Remember that the JVM is not the only application on the server that will be using the RAM. Take care to leave some RAM available for other processes.

(6) Edit the AUTOEXEC.BAT file found at the root of the C:\ drive. It should look similar to the example below:

C:
CD \NWSERVER
SERVER

(7) After editing the file to add the -U switch, it should look similar to the example below, with the exception that the number value may be different depending on the desired amount of RAM needed for the JAVA process.

C:
CD \NWSERVER
SERVER -U1000000000

** NOTE ** In this example we are specifying 1 GIG of RAM to be used. The parameter is measured in bytes. This will get past the limit specified in step (4). You should now be able to increase your heap size above 400 MEG.

Additional Information

This is caused from Tomcat needing more memory to run memory intensive web applications such as Novell Portal Services 1.5. This does not mean that you are out of RAM. This is a common misconception. You simply need to adjust your minimum and maximum heap size for tomcat to use. This will allow Tomcat to use more memory.
Another example of how the server -U switch was used to resolve the symptoms described above:

The Portal server had 2GB of physical memory. Keeping the"use 3/4 of the total physical memory available rule" in mind, we configured the server to use
-U=1500000000 and the TOMCAT4.NCF file to load java with -Xmx1400M

** NOTE ** we are specifying 1.5GB of physical memory, a maximum heap size of approximately 1.4GB, and are not specifying a minimum heap size.

When monitoring the server memory, the servers would reach approximately 800MB to 900MB before the JVM garbage collection process would kick in to clean up JAVA and set the heap back to around 383MB. This completely resolved the out of memory errors, slow applications and inability to login to the application.

** NOTE ** One thing to be aware of is that if you do a RESTART SERVER command from the console, the -U setting does not hold with the kernel. You must use the RESET SERVER command in order to keep the -U option.

Formerly known as TID# 10070512