Novell Home

AppNote: Shimming CLib Context into NetWare Loadable Modules (NLMs) Built for LibC

Novell Cool Solutions: AppNote
By Russell Bateman

Digg This - Slashdot This

Posted: 2 Dec 2004

Russell Bateman
Senior Software Engineer
Server-library Development

This AppNote discusses how to write LibC-based NLMs that are able to consume CLib interfaces in the NLM environment.


CLib Context
Brokering CLib Context for a LibC NLM
Sample Code
Topics Library development, NLM development, kernel code, CLib, context
Products NetWare 5.1, NetWare 6, NetWare 6.5
Audience Developers
Level Advanced
Prerequisite Skills   Familiarity with CLib and LibC programming
Operating System      NetWare 5 and above
Tools NDK
Sample Code Yes


There are numerous CLib-based components, including non-Novell ones that have not been ported to LibC. As LibC imposes itself as the preeminent execution environment for NetWare and more and more applications port to this environment, difficulties arise because these important CLib-based applications and services aren't callable from LibC applications. LibC threads (and many other types) cannot call into a CLib-based component if they then find themselves in CLib itself since they do not have the necessary context. The result is anything from a page fault to failure to mysterious success.

libcclib.nlm, also referred to as the context broker, is a multiloadable NLM that shims CLib context support for threads from LibC-based and other NLMs whose threads wander into CLib and therefore need its context. libcclib.nlm brokers CLib context and is loaded separately by each consuming NLM application. This document discusses the interfaces and interface methods of an approach to solving the CLib context problem.

CLib Context

General purpose operating systems provide a user space in which applications run. While applications hosting was not an afterthought on NetWare, the original intention was that applications would constitute mostly kernel extensions. In the days of NetWare 386, the idea of a pure network operating system was easily accepted. The day would come in which it would become important to host applications even on a file and print server as file and print services would themselves become commoditized. But, that day hadn't come by the advent of NetWare 386.

As soon as putting standard C applications was thought of, two impediments were realized. First, there was no standard runtime library available to furnish printf, fopen, strcpy, etc. Second, in contemplating creating such a library, it became immediately obvious that there were going to be problems of data instancing that would be very difficult to solve in a kernel-extension only environment. Enter the phenomenon known as "CLib context."

The NetWare kernel originally had no ability to erect boundaries around applications. A global variable such as timezone in the library, would be visible to all applications and any application that changed the value of this variable for itself would also change it for all other applications, to dramatic and sometimes catastrophic effect.

In order to solve this problem, indeed in order to create a partition between NLM applications which shared a single address space with the operating system itself, it was necessary to put together and manage a little operating kernel. This CLib did by imposing the requirement on participating applications that they link a special prelude object. This prelude object would set up global instance data and create lists of resource ownership that would be respected by calls into the library for standard C runtime interfaces, particularly those that had an effect upon instanced data values and other resource states.

Originally, the notion that some applications, generally those written by third parties and some bigger ones internal to Novell, would be "CLib NLMs." However, most NLMs thought more of as "kernel extensions" like drivers, namespace implementations, libraries, etc. would not be CLib NLMs. The latter would instead forgo the useful services of a C runtime library in favor of coding whatever they needed for themselves plus they would not need instance data since, for the most part, their developers were in a kernel-level, multithreaded, networking mindset.

In order for so-called CLib NLMs to be able to call such functions as read (after calling open) or to set errno to be preserved only for the calling thread, etc., CLib had to erect a stringeant set of data structures. This it did only for those NLMs that linked its prelude object, originally called prelude.obj, and when a thread came into CLib that didn't have the appropriate context markings, the result was undefined, but quite often catastrophic.

More recently, LibC has been architected to be very permissive of threads calling it without sufficient or meaningful context. For example, if a system thread, like the one that maintains the NetWare command console, were to make a call to read passing descriptor value 46, this function would merely set the calling thread's errno to ENOCONTEXT, and safely return without page faulting (barring any attempts made by the clown who caused this nonsensical situation in the first place to befuddle the library even more by duping it into thinking that the system thread actually belonged to some NLM with an open descriptor table). In such a situation CLib would probably attempt to access a non-existant open descriptor table–a process that would lead to a page fault.

Why CLib Context?

All this begs the question: Why would an NLM need CLib context if it is written to LibC in the first place?

LibC was written more to cater to the needs of multithreaded and Open Source applications being ported to NetWare. It was the Novell solution to increased application platform support. Some "NetWare-ish" interfaces don't port easily to UNIX. In a sense LibC owes increasingly more to UNIX than to CLib since it attempts to mimic POSIX and UNIX-style programming support. In many cases, CLib's NetWare-specific functionality which lies outside the simple C runtime expectations hasn't reached LibC yet, in others, it just doesn't fit what LibC is trying to do. In still other cases, it is flatly impossible. Some applications on NetWare want to take advantage of what LibC offers, but still need to consume interfaces that only CLib can provide. The method discussed in this article is a sort of last resort. In order to call CLib safely for this support, a LibC thread has to have CLib context. If done carefully, this sort of context brokering can solve most simple problems of CLib context. This article is precisely about how to set up this sort of context.

First, an important note about the source code we are using...

libcclib.nlm was written as a multiple-load NLM meaning more than one copy of it can be loaded and resident simultaneously. This would not necessarily be the case of a real, third-party version as that version would not ordinarily support more than one application. If libcclib.nlm were to ship as a product, then it would have to be designed as it is since it would need to support myriad third-party products without knowing their particulars. The reason that libcclib.nlm does not ship as a supported Novell product is because context brokering isn't always as simple as it appears on the surface nor even as solvable a problem as this article will make it out to be. Novell would be naïve to assume that libcclib.nlm could solve the particular problems of every NLM needing brokering. Thus, this NLM has only been communicated to developers requesting it principally as source in addition to a binary to try out. Prior to this article, the technology has only been available upon request.

It would be inappropriate for developers to use the exported interface names in libcclib.nlm as advertised here. There are three different ways of renaming these interfaces to avoid conflict with this NLM and other context-brokering NLMs based on libcclib.nlm including a) choosing new names altogether, b) prefixing these interface names and c) using these names plus linker-added prefixes. The second simply means that the individual broker keeps the same names as in this article, but inserts a product-specific prefix in front them. The third refers to the practice of instructing the NLM linker to prefix the names with the NLM name in uppercase plus an at-sign.

Loading and Unloading the Context Broker

The following section refers to the implementation of libcclib.nlm and not to how a private implementation would work. A private implementation must find its own way of launching its context broker. This implementation, for demonstration purposes, relies on help from CLib.

The context broker is loaded by calling CLibLoadContextBroker exported by CLib. This interface is basically a call to the NetWare Loader to load libcclib.nlm with a special argument. Before the calling NLM allows itself to be unloaded, it should unload the broker explicitly by calling CLibUnloadContextBroker.

A third-party context broker must emulate this process by providing similar calls. In the following description and discussion of code, libcclib.nlm refers in fact to the third-party context broker which will not have this name.


   #include <libcclib.h>

   int CLibLoadContextBroker( void *module,
	   const char *callback );
   int CLibUnloadContextBroker( clibctx_t *broker );


The CLibLoadContextBroker function loads a copy of libcclib.nlm to broker CLib context for the calling NLM, which must pass its NLM module handle (from LibC's getnlmhandle) and the name of a function it exports. The sample prototype for this function, which cannot collide in name with that exported by any other NLM loaded at the same time, is:

        int MyBrokerCallBack( clibctx_t *broker );

The CLibUnloadContextBroker function unloads the copy of libcclib.nlm loaded by the previous call to CLibLoadContextBroker.


The CLibLoadContextBroker function returns 0 upon success. If it fails, it returns an error. The CLibUnloadContextBroker function returns 0.


If an error condition occurs, the CLibLoadContextBroker function returns the corresponding value in order of likelihood of occurrence. CLibUnloadContextBroker does not return an error.

ENOENT libcclib.nlm could not be found (installation error)
ENOMEM Insufficient memory was available to load libcclib.nlm
EWRNGKND The format of libcclib.nlm was not that of an executable NLM binary
ESERVER Some error occurred in the NetWare Loader and libcclib.nlm could not be loaded
EINUSE libcclib.nlm was linked defectively and cannot be loaded multiple times
EAGAIN libcclib.nlm is already being loaded
EEXIST libcclib.nlm exports symbols that already exist

Source code (not included in libcclib.nlm sources):

int CLibLoadContextBroker
      // load-definition structure of calling NLM
      void      *module,
      // call-back in (calling) NLM seeking brokerage
      const char   *cb_name
      char      commandline[256];

      ** Load the LibC-calling-CLib context broker. This is a
      ** multiloadable shim that exports a function interface via
      ** calling back into the NLM that loaded it. It imports the
      ** call-back name, cb_name, and then calls through that to
      ** deliver a pointer to a structure that contains its
      ** interfaces (see clibctx_t in libcclib.h).
      strlist(commandline, "LIBCCLIB.NLM", " ", cb_name, NULL);
      return 0;

int CLibUnloadContextBroker
      clibctx_t      *ctx
      return 0;

Using the Context Broker

When the context broker has loaded and initialized itself, it calls back the NLM that loaded it via the function named in the call to CLibLoadContextBroker. That function, called MyBrokerCallBack here for the sake of having a name (it could be called anything that isn't already being exported publicly from the calling NLM), receives from libcclib.nlm a pointer to structure clibctx_t. This structure contains interfaces into the calling NLM's instance of libcclib.nlm. The call-back function must allocate its own copy of this structure and initialize it identically as the one whose pointer is passed as an argument. Then it must return 0. Any other value returned will cause the loaded copy of libcclib.nlm to error out and unload.

typedef struct
   int  (*ThreadGroupGetID)  ( void );

   int  (*ThreadGroupCreate) ( const char *name,
                              int *threadGroupID );
   int  (*ThreadGroupDispose)( int threadGroupID );
   int  (*ThreadGroupUnwrap) ( int threadGroupID,
                              int restoredThreadGroupID );
   int  (*ThreadGroupWrap)   ( int threadGroupID );

   int  (*ThreadCreate)      ( int threadGroupID,
                              void (*start_routine)( void *arg ),
                              void *arg, size_t stackSize,
                              unsigned long flags,
                              int *threadID );
   void *reserved[20];
} clibctx_t;

When the application NLM (that loaded libcclib.nlm) wants to create context, wrap context on a thread or unwrap context from a thread, etc., it calls through the function pointers in the clibctx_t structure as explained in the synopses of these various interfaces below.



   int ThreadGroupCreate( const char *name, int *threadGroupID );


The ThreadGroupCreate function causes a new CLib thread group to be created with no thread and returns its ID for use in passing to ThreadGroupWrap. Argument name is only used briefly during execution in debug mode. threadGroupID holds the ID of the CLib thread group after a successful return.


The ThreadGroupCreate function returns 0 upon success or -1 upon failure. No explicit information is available (due to the paucity of CLib's BeginThreadGroup error handling), but any error is most likely a failure to allocate memory.

Upon return, ThreadGroupCreate has created a thread group with no running thread in it. This is achieved because (look at the code) CLib's SetThreadGroupID function behaves specially when it is passed a 0: the thread group is stripped off the calling thread. Since, in the implementation here, the thread only runs to completion of the call to SetThreadGroupID, the thread dies leaving a naked thread group.

If a thread is brought into this new context (via ThreadGroupWrap) and then exits before restoring its prior context (via ThreadGroupUnwrap), then this thread group will go away also. Should other threads be using it, it will not go away until they do, but once it has gone away, the thread group identity, threadGroupID, becomes stale and unusable.


   #include <libcclib.h>

   int ThreadGroupDispose( int threadGroupID );


The ThreadGroupDispose function frees up the new CLib thread group after it is no longer needed. This function doesn't really dispose of the thread group which persists until every thread belonging to it exits. It is the responsibility of the calling application to dispose of such threads. Note that it is possible that the thread wrapped in this context has itself caused the creation of others outside the calling application's control. Nevertheless, this should not result in a fault or permanent resource leak at unload.


The ThreadGroupDispose function returns 0 upon success or ENOMEM on failure. Failure of this function is not serious, but only implies resource leaking as long as the calling NLM remains loaded.


   #include <libcclib.h>

   int ThreadGroupWrap( int threadGroupID );
   int ThreadGroupUnwrap( int threadGroupID,
		int restoredThreadGroupID );


The ThreadGroupWrap function wraps the calling thread with CLib context and adopts it into the thread group specified by threadGroupID.

The ThreadGroupUnwrap function undoes this wrapping and, potentially, retores a different thread group ID where called for. It isn't impossible for a thread to have specific thread group context on it before calling ThreadGroupWrap. If this is the case, it is preferable to restore this context rather than to return the thread to a context of 0 (system or LibC thread). threadGroupID will be checked against the actual current thread group ID and, if found not to match, error EINVAL will be returned. This is to guard against nonsensical use of the wrapping interfaces by the calling NLM. The ThreadGroupUnwrap function does not need to be called as context can simply be left on the thread for the duration of its life.


The ThreadGroupWrap function, like CLib's SetThreadGroupID, merely returns -1 on failure. On success, it returns the former thread group ID.

The ThreadGroupUnwrap function returns 0 upon success or ENOMEM on failure. Failure of this function is not serious, but only implies resource leaking as long as the calling NLM remains loaded.


   #include <libcclib.h>

   int ThreadGroupGetID( void );


The ThreadGroupGetID function returns the current thread group ID of calling thread if any. It is identical to CLib's GetThreadGroupID.


The ThreadGroupGetID function, like CLib's GetThreadGroupID, merely returns -1 on failure. On success, it returns the thread group ID of the calling thread.

Building your context broker

clibpre.c - You don't have to use this file in your project if you don't want to; you can just link in prelude.o, nwpre.o or clibpre.o from the NDK.

libcclib.c -- This file contains the essential code we have been discussing including the main function of the context broker NLM. You should rename the file to help you remember not to keep the same names inside. Remember: your context broker is nothing more or less than a very simple CLib NLM.

clib.h -- This file contains quick and dirty prototypes so that our project doesn't need to include a lot of headers from the CLib NDK. It is unnecessary and you can just include the usual CLib headers if you want.

libc.h -- This file need not exist either. There are pure CLib equivalents for getnlmhandle and Import/ExportPublicObject if you want.

clibctx.h -- This file should be rewritten with the names of your own prototypes instead of the ones chosen by LibCCLib. It doesn't even have to retain this name of course. It does contain an external data reference (gMainThreadGroupID) that might not need to be visible outside of your equivalent to libcclib.c.

libcclib.h -- Similarly, this file should be renamed. The clibctx_t structure is generic. Though it points to functions whose names you should modify, the actual field names do not need this treatment. Remember: it is a pointer to this structure in the broker that your consuming NLM gets by which it consumes the various functions to broker context.

Please consult the source code distributed for libcclib.nlm. Feel free to copy from it and rearrange it according to your needs.

Sample NLMs using brokered context

Our sample here consists of two NLMs. First, a LibC NLM that calls into a second NLM that itself uses CLib. In order to call into the second NLM, the first needs to call the broker, libcclib.nlm, for context to do so. The reason I create the second NLM instead of making calls directly into CLib in the first one is because it's very hard to compile code making calls into LibC and needing LibC headers as well as into CLib and needing CLib headers most of which conflict in name and in content with LibC's.

The NLM prompting you to write a context broker is like the first NLM here, libcnlm.nlm. The second NLM, clibnlm.nlm, may not exist in your world if you go to the trouble of creating a hybrid LibC-CLib compilation environment. I find this too much bother, so I created a second NLM.


#include <stdio.h>
#include <screen.h>
#include <stdlib.h>
#include <string.h>
#include <library.h>
#include <libcclib.h>
#include <semaphore.h>

sem_t       sema;

extern void DoCLibStuff( void );

clibctx_t   gCLibCtx;

int         LibCNLMBrokerCallBack( clibctx_t *ctx );

int LibCNLMBrokerCallBack
      clibctx_t	*ctx
      gCLibCtx = *ctx;
      return 0;

void main( void )
      int	err, tgID;
      char	tgStr[16];
      void	*h;

      if (sem_init(&sem, 0, 0))
            printf("Failed to create new semaphore!\n");

      ** Create a LibC-based NLM that calls into a CLib-based NLM
      ** to demonstrate that our context brokering agent,
      ** libcclib.nlm, works.
      h   = getnlmhandle();
      err = CLibLoadContextBroker(h, "LibCNLMBrokerCallBack");

      if (err)
            consoleprintf("Failed to load libcclib.nlm: %s\n",


      err = (*gCLibCtx.ThreadGroupCreate)("libcnlm", &tgID);

      if (err)
            consoleprintf("Failed to create new thread group: "
                              "%s\n", strerror(err));

      err = (*gCLibCtx.ThreadGroupWrap)(tgID);

      if (err)
            consoleprintf("Failed to wrap new thread group: "
                              "%s\n", strerror(err));

      itoa((*gCLibCtx.ThreadGroupGetID)(), tgStr, 10);
      consoleprintf("Using thread group %s...\n", tgStr);

      // make calls into a CLib-based NLM here...

      err = (*gCLibCtx.ThreadGroupUnwrap)(tgID, 0);

      if (err)
            consoleprintf("Failed to unwrap new thread group: "
                              "%s\n", strerror(err));

      (void) (*gCLibCtx.ThreadGroupDispose)(tgID);



#include <stdio.h>
#include <nwthread.h>

void DoCLibStuff( void );

void DoCLibStuff( void )
      // this NLM is like a CLib-based library calling into CLib
      printf("This is a test to see if CLibNLM.NLM was called "
                        "with context...\n");

void main
      int   argc,
      char   *argv[]
      ExitThread(TSR_THREAD, 0);
Download, which includes the following:


Download, which includes libcclib.nlm and additional files referenced in this document.

Novell Cool Solutions (corporate web communities) are produced by WebWise Solutions.

© 2014 Novell