Sentinel 6.1 das_binary dumps core repeatedly after installation

  • 7005202
  • 22-Jan-2010
  • 11-Jun-2013

Environment

Novell Sentinel 6.1

Situation

While Sentinel is running the $ESEC_HOME/log directory is filled with memory dumps from the das_binary process.  This happens while the system is under a load with regard to database insertions and also when the DAS services are installed on a system with fairly low amounts of RAM (2 GB) which does not meet Sentinel's requirements.  The das_binary process is automatically restarted and continues to work but may continue running out of memory and writing out a core unless the environment is changed.

Resolution

The default Sentinel 6.1 installation configures the maximum amount of heap available for each Sentinel process based on the system's total RAM.  Systems with more RAM have more memory assigned per the Xmx values for the various Sentinel Java processes. Thus the more RAM available at install the higher the Xmx values and the avoidance of this issue.

A system with the minimum requirements per the Sentinel documentation has very little RAM to give to each Sentinel process including das_binary.  As a result if a significant load is put on das_binary the buffers within that process may accept more data from the message bus than the containing Java environment will allow.  The result is that Java dumps its memory to the memory dump files in $ESEC_HOME/log.  The Sentinel "wrapper" will automatically restart das_binary in order to keep the system running.  The multiple das_binary starts will also show up in the Sentinel Control Center (SCC) under the Admin tab, Servers View.

The easiest way to resolve this is to increase the Xmx parameter in the $ESEC_HOME/config/configuration.xml file to give das_binary additional RAM for its large buffers.  This must be done with the physical capabilities of the server in mind.  Typically increasing the specified Xmx value from 160 MB to 320 MB is okay and will not over-allocate the system's memory resources.  Avoiding the use of the system's virtual memory (swap) space is very highly encouraged due to the performance requirements of Sentinel and the performance limitations imposed by using virtual memory.

Example of the value to change for das_binary in the configuration.xml file, values are located towards the end of the file:

Example before change:
process component="DAS" depends="UNIX Communication Server,Windows Communication Server" image=""$(ESEC_JAVA_HOME)/java" -server -Dsrv_name=DAS_Binary -Xmx160m -Xms64m -XX:+UseParallelGC -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=../log/DAS_Binary.hprof -Xss136k -Xrs -Duser.language=en  -Djava.net.preferIPv4Stack=true -Dfile.encoding=UTF8 -Desecurity.cache.directory=../data/das_binary.cache -Desecurity.dataobjects.config.file=/xml/BaseMetaData.xml -Djava.util.logging.config.file=../config/das_binary_log.prop -Dcom.esecurity.configurationfile=../config/configuration.xml -Djava.security.auth.login.config=../config/auth.login -Djava.security.krb5.conf=../config/krb5.conf -jar ../lib/ccsbase.jar ..//config//das_binary.xml" min_instances="1" name="DAS_Binary" post_startup_delay="20" type="container" working_directory="$(ESEC_HOME)/data"/>


Example after change:
process component="DAS" depends="UNIX Communication Server,Windows Communication Server" image=""$(ESEC_JAVA_HOME)/java" -server -Dsrv_name=DAS_Binary -Xmx320m -Xms64m -XX:+UseParallelGC -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=../log/DAS_Binary.hprof -Xss136k -Xrs -Duser.language=en  -Djava.net.preferIPv4Stack=true -Dfile.encoding=UTF8 -Desecurity.cache.directory=../data/das_binary.cache -Desecurity.dataobjects.config.file=/xml/BaseMetaData.xml -Djava.util.logging.config.file=../config/das_binary_log.prop -Dcom.esecurity.configurationfile=../config/configuration.xml -Djava.security.auth.login.config=../config/auth.login -Djava.security.krb5.conf=../config/krb5.conf -jar ../lib/ccsbase.jar ..//config//das_binary.xml" min_instances="1" name="DAS_Binary" post_startup_delay="20" type="container" working_directory="$(ESEC_HOME)/data"/>

This defect does not exist in later versions of the product so this defect has been closed as Will Not Fix.  Please upgrade to the current version.