NCP engine coring in LookupUTF8 or ConnInfoRPCHandler.

  • 7015267
  • 23-Jun-2014
  • 04-Mar-2015

Environment

Novell Open Enterprise Server 11 (OES 11) Linux Support Pack 1
Novell Open Enterprise Server 11 (OES 11) Linux Support Pack 2

Situation

Novell OES11 SP1 server is infrequently coring in NCP engine.
Novell OES11 SP2 server is infrequently coring in NCP engine.

Resolution

Reallocate the reply buffer if the required size exceeds a fixed size of 64k.

The fix has been publicly released with the respective January 2015 OES Scheduled Maintenance patches for OES11 SP1 and OES11 SP2.

Cause

The ConnInfoRPCHandle() handlers sends various connection info to the caller (e.g. 'ncpcon') including the open files list. While replying to RPCs the server responds in a default buffersize of 64k. However in this case the responded size was around 91K, because of which BuildConnectionInformationReply() called in SendConnectionInformation() overwrites subsequent memory locations beyond what was allocated to replyBuffer. That is spread across other memory variables like 'openFiles' etc. At the end while trying to free 'openFiles' it cores because of this.

Additional Information

Back trace of a crash observed on OES11 SP1 code as below :
(gdb) bt
#0  0x00007f82b56b19fc in LookupUTF8(VolumeCacheData*, unsigned int, unsigned
char*, unsigned long) ()
   from /opt/novell/eDirectory/lib64/nds-modules/libncpengine.so
#1  0x00007f82b56b402e in GetEntryFromDirCache(unsigned int, int, unsigned int,
int, unsigned char*, int, stat*, CacheEntry**)
    () from /opt/novell/eDirectory/lib64/nds-modules/libncpengine.so
#2  0x00007f82b56b94f6 in GetAllEntryInfoFromDirCache(unsigned int, int,
unsigned int, int, unsigned char*, int, CacheEntryInfo*, stat*, unsigned int*,
unsigned int) () from /opt/novell/eDirectory/lib64/nds-modules/libncpengine.so
#3  0x00007f82b56d8148 in DirectorySearch(unsigned int, int, unsigned int, int,
unsigned int, unsigned char*, int, int, pseudo_netware_direntry*, unsigned
int*, char*, char*, CacheEntryInfo*, stat*) ()
   from /opt/novell/eDirectory/lib64/nds-modules/libncpengine.so
#4  0x00007f82b56cce96 in ContinueFileSearchSet(unsigned int, int, unsigned
int, unsigned int, int, unsigned int, int, char*, unsigned int, unsigned int*,
unsigned char*, unsigned short*, char*, int*, int) ()
   from /opt/novell/eDirectory/lib64/nds-modules/libncpengine.so
#5  0x00007f82b56e30e4 in Case89(unsigned int, int, svc_request*, int) ()
   from /opt/novell/eDirectory/lib64/nds-modules/libncpengine.so
#6  0x00007f82b56f7467 in ExecuteNCPPacket(unsigned int, svc_request*, int) ()
   from /opt/novell/eDirectory/lib64/nds-modules/libncpengine.so
#7  0x00007f82b56c7fc3 in INCP::HandleNCPFileServiceRequest() () from
/opt/novell/eDirectory/lib64/nds-modules/libncpengine.so
#8  0x00007f82b56c9c45 in INCP::Process(int, void (*)(void*, int, int, unsigned
long, void const*, int (*)(void*, int, unsigned char, unsigned int, ...))) ()
from /opt/novell/eDirectory/lib64/nds-modules/libncpengine.so
#9  0x00007f82b56c9f3b in INCP::HandleNCPRequest(ReceiveBufferStruct*, int,
int*) ()
   from /opt/novell/eDirectory/lib64/nds-modules/libncpengine.so
#10 0x00007f82b56cac1b in
INCP::ServiceStreamGroupConnections(StreamGroupStruct*) ()
   from /opt/novell/eDirectory/lib64/nds-modules/libncpengine.so
#11 0x00007f82b56cb2ba in NCPPollerThread(StreamGroupStruct*) () from
/opt/novell/eDirectory/lib64/nds-modules/libncpengine.so
#12 0x000000000041737c in ?? ()
#13 0x00007f82b87317f6 in start_thread () from /lib64/libpthread.so.0
#14 0x00007f82b7cf3f8d in clone () from /lib64/libc.so.6
#15 0x0000000000000000 in ?? ()
#


It was determined that another crash in NCP Engine on OES11 SP2 code with the following characteristics had the same root cause
oes11 kernel: [1542812.948859] ndsd[3739] general protection ip:7f3e127d4661 sp:7f3e136459b0 error:0 in libncpengine.so.0.0.0[7f3e12782000+11a000]
oes11 kernel: [1544065.305820] ndsd[28107] general protection ip:7fbec5ac24a4 sp:7fbea7fe6f50 error:0 in libtcmalloc_minimal.so.0.2.1[7fbec5aa4000+27000]
Back trace of a crash observed on OES11 SP2 code as below :
#bt
#0  0x00007fbec5ac24a4 in tc_malloc () from /usr/lib64/libtcmalloc_minimal.so.0
#1  0x00007fbec57d2142 in ?? () from /opt/novell/lib64/libccs2.so
#2  0x00007fbec581efa7 in ?? () from /opt/novell/lib64/libccs2.so
#3  0x00007fbec57d0f2f in ?? () from /opt/novell/lib64/libccs2.so
#4  0x00007fbec57d2b7e in ?? () from /opt/novell/lib64/libccs2.so
#5  0x00007fbec57bb993 in ?? () from /opt/novell/lib64/libccs2.so
#6  0x00007fbec57afe10 in ?? () from /opt/novell/lib64/libccs2.so
#7  0x00007fbec578ce39 in ?? () from /opt/novell/lib64/libccs2.so
#8  0x00007fbec5768d40 in ?? () from /opt/novell/lib64/libccs2.so
#9  0x00007fbec57674a0 in ?? () from /opt/novell/lib64/libccs2.so
#10 0x00007fbec57675a8 in ATX_EncryptWithPrivateKey () from /opt/novell/lib64/libccs2.so
#11 0x00007fbec06f583d in ATEncryptWithPrivateKey () from /opt/novell/eDirectory/lib64/nds-modules/libnds.so.1
#12 0x00007fbec04a0a21 in EncryptWithPrivateKey(char*, unsigned long, char*, char**) () from /opt/novell/eDirectory/lib64/nds-modules/libnds.so.1
#13 0x00007fbec04a3787 in DSABeginAuthentication(unsigned long, char*, unsigned long, unsigned long*, char**) ()
   from /opt/novell/eDirectory/lib64/nds-modules/libnds.so.1
#14 0x00007fbec04d2fd1 in DSDummyVerbRedirector(int, int, int, unsigned long, char*, unsigned long, unsigned long*, char**) ()
   from /opt/novell/eDirectory/lib64/nds-modules/libnds.so.1
#15 0x00007fbec04d63da in DSACommonRequest(int, int, unsigned int, int, char*, unsigned long, char*, unsigned long, unsigned long*, char**) ()
   from /opt/novell/eDirectory/lib64/nds-modules/libnds.so.1
#16 0x00007fbec04d69d8 in DSAWireRequest(unsigned int, int, int, char*, unsigned long, char*, unsigned long*, char**) ()
   from /opt/novell/eDirectory/lib64/nds-modules/libnds.so.1
#17 0x00007fbec06eddb3 in ?? () from /opt/novell/eDirectory/lib64/nds-modules/libnds.so.1
#18 0x00007fbec06ee9e8 in ?? () from /opt/novell/eDirectory/lib64/nds-modules/libnds.so.1
#19 0x00007fbec06eccac in ?? () from /opt/novell/eDirectory/lib64/nds-modules/libnds.so.1
#20 0x00007fbec13ce590 in INCP::Process(int, void (*)(void*, int, int, unsigned long, void const*, int (*)(void*, int, unsigned char, unsigned int, ...))) ()
   from /opt/novell/eDirectory/lib64/nds-modules/libncpengine.so
#21 0x00007fbec13d023f in AsyncNCPThread(AsyncNCPStruct*) () from /opt/novell/eDirectory/lib64/nds-modules/libncpengine.so
#22 0x000000000041a072 in PoolWorker (data=0x106fee40) at /usr/src/debug/novell-NDSbase-8.8.8.2/nds-8.8.8.2/unix/dhost/ddstpool.cpp:402
#23 0x00007fbec444c806 in start_thread () from /lib64/libpthread.so.0
#24 0x00007fbec3a12cad in ?? () from /lib64/libc.so.6
#25 0x0000000000000000 in ?? ()
#