Handling ndsd (eDirectory) core files on Linux and Solaris
This document (3078409) is provided subject to the disclaimer at the end of this document.
Environment
Novell eDirectory 8.7.3 for Linux
Novell eDirectory 8.7.3 for Solaris
Novell eDirectory 8.8 for Solaris
Situation
eDirectory 8.7.3 /var/nds/dib
eDirectory 8.8.x /var/opt/novell/eDirectory/data/dib
If ndsd is halted and the reason is not apparent, check for a core file in the dib directory. If no core file is present, change the ulimit parameter:
ulimit -c 10000000
This will allow core files up to 10 GB.
Note: Many Linux distributions set the ulimit value to '0' in /etc/profile or use 'ulimit -Sc 0' to prevent core files. If problems are recurring, set this parameter in /etc/profile and remove/comment out the current settings.
Note: It is not possible to generate a core in eDirectory on Linux using SIGQUIT.
Resolution
Novell-getcore is script used to gather and bundle the ndsd core file and all necessary libraries needed to read the core file. Novell-getcore is installed as part of the NDSserv package, beginning with eDirectory 8.7.3.9 and eDirectory 8.8.2. If you have an earlier eDirectory version, the novell-getcore script can be downloaded from http://download.novell.com
Using novell-getcore to gather a core and related libraries
1) Verify GDB is installed on the eDirectory server by typing "gdb -version". GDB is required to be installed prior to using novell-getcore.
2) Create a bundle with novell-getcore to send to Novell Technical Support:
eDirectory 8.7.3 example:
novell-getcore -b /var/nds/dib/core.#### /usr/sbin/ndsd
eDirectory 8.8 example:
novell-getcore -b /var/opt/novell/eDirectory/data/dib/core.#### /opt/novell/eDirectory/sbin/ndsd
(where ####, is the PID number of ndsd when it cored)
3) On Linux: Grab a supportconfig from this server. If you need the script it can be downloaded from the following page: http://www.novell.com/communities/node/2332/supportconfig-linux
On Solaris: Use unixinfo to create a unixinfo.log. See TID 10075466 How to create a UNIX configuration file.
4) Upload the supportconfig or unixinfo.log and novell-getcore bundles to ftp.novell.com/incoming
NOTE: Sometimes the reason ndsd cores is due to memory corruption. If this is the case, it is helpful to have malloc core ndsd at the time the corruption occurs so that the module that caused the corruption can more easily be detected in the core.
The novell-getcore script extracts the stack into the file ./tmp/novell-getcore_gdb_command_output.####. The section with the stack will start at GDBCMDSTART bt and end with GDBCMDEND bt.
When there is stack damage present, the debugger (gdb) can't resolve the stack pointers to a function
This is an example of a stack that has been corrupted and the debugger is unable to resolve the stack pointers:
# GDBCMDSTART bt
#0 0xb445b8cf in ldap_tls_inplace () from /usr/lib/libldap-2.3.so.0
#1 0xb445e1b2 in ldap_install_tls () from /usr/lib/libldap-2.3.so.0
#2 0xb446c2b6 in ?? () from /lib/libnss_ldap.so.2
#3 0x0807fa48 in ?? ()
#4 0x0807fdf0 in ?? ()
#5 0x00000001 in ?? ()
#6 0xb6490668 in ?? ()
#7 0xb6490670 in ?? ()
#8 0xb44680db in ?? () from /lib/libnss_ldap.so.2
#9 0xb4467000 in ?? ()
#10 0x000135d8 in ?? ()
#11 0xb7f6b8c8 in ?? () from /opt/novell/eDirectory/lib/libldapsdk.so.0
#12 0xb447a4e0 in ?? () from /lib/libnss_ldap.so.2
#13 0xb64906e4 in ?? ()
#14 0x0000001e in ?? ()
#15 0x00000000 in ?? ()
# GDBCMDEND bt
If ndsd cores due to stack corruption, Novell Technical Support will request an ndsd core be submitted where the environment variable MALLOC_CHECK_=2 is active for ndsd.
To set this environment variable:
Edit the /etc/init.d/pre_ndsd_start script, add the following and restart ndsd.
export MALLOC_CHECK_=2
## Note in eDirectory 8.8.5 ftf2 (patch2) the location of the pre_ndsd_start has been moved from /etc/init.d to /opt/novell/eDirectory/sbin/. It should also be noted that this setting can also be added directly in to the /etc/init.d/ndsd script. The contents of the pre_ndsd_start script are sourced into ndsd at the time ndsd starts. Be aware that any permanent settings (MALLOC_CHECK_=2 should not be permanent! ) will be overwritten if left in the ndsd script the next time an eDirectory patch is applied, the pre_ndsd_start script will not be touched and in this lies the purpose of this file.
To verify this ndsd environment variable is set properly while ndsd is running, do the following:
strings /proc/`pgrep ndsd`/environ | grep -i MALLOC_CHECK_"
After ndsd has cored, to verify the core file had the ndsd environment variable set, do the following:
strings core.#### | grep -i MALLOC_CHECK_"
Bundle the core with MALLOC_CHECK_=2 set as in step 2.
MALLOC_CHECK_=2 should only be used when troubleshooting cores related to possible memory corruption. This environment variable can have a performance impact on some systems due to the increased memory checking being done by malloc. In eDirectory 8.8, it will cause ndsd to revert back to using malloc instead of tcmalloc_miminal which was added to enhance performance.
Another side effect of using MALLOC_CHECK_=2 is the possibility of increased coring. Malloc will cause ndsd to core whenever a memory violation is detected whether or not it would have caused ndsd to crash under normal running conditions.
Status
Diagnostic Pattern Available through Novell Support AdvisorAdditional Information
Have you tried resolving this issue using Novell Support Advisor? Visit http://support.novell.com/advisor/ to learn more.
Document
| Document ID: | 3078409 |
| Creation Date: | 01-09-2008 |
| Modified Date: | 01-31-2010 |
| Novell Product: | eDirectory |
Disclaimer
The Origin of this information may be internal or external to Novell. Novell makes all reasonable efforts to verify this information. However, the information provided in this document is for your information only. Novell makes no explicit or implied claims to the validity of this information.
Any trademarks referenced in this document are the property of their respective owners. Consult your product manuals for complete trademark information.