Novell Home

How to Run Binary-Only Application Packages on Various Versions of Linux

Novell Cool Solutions: Feature

Digg This - Slashdot This

Posted: 3 Mar 2005
 

Introduction
Background
The Scenario
Diagnosing Problems
The Solution
Step 1
Step 2
Step 3
Additional Problems
Conclusion

Introduction

The purpose of this document is to provide a migration path for users who have invested in proprietary application packages that were certified for specific versions of various basic libraries (such as glibc) but who now intend to run them on different versions.

The reader of this document should be able to install and configure the files required for such a task. (The document will also provide several hints.) All examples are based on x86 hardware; other Linux architecture changes may require changes to both to the example code and the recipes.

This document is the result of using the described steps in a real project.

Background

A Linux program usually consists of machine code to perform a specific set of functions. Normally, some of these functions are provided by libraries. A library can exist as a static version included in an archive, or it can exist as a dynamic, shared object. When a program is created, the developer decides whether static or shared libraries will be used. There are only a few instances in which a static library should be used; such instances include binary incompatibilities between different versions of a library or evolving changes in the infrastructure of such a library.

A typical example of the latter is what occurred while the programming language C++ matured but still wasn't commonly standardized. Implementation details such as how virtual functions in derived inheritance lattices are addressed resulted in different, incompatible versions of the same library, depending on the version of the compiler that was used to create the library.

A shared library has several benefits: its machine code is loaded only once into the computer's memory, it is shared among several programs using this library and only the memory for program-specific data needs to be allocated. Another benefit is that a system administrator can easily upgrade a shared library without having to touch any programs using that library; this is especially important for security fixes.

Of course, none of this is Linux-specific; all of these descriptions also apply to other modern operating systems that support shared libraries.

The Scenario

Remember when you wanted to execute a newly downloaded program on your brand-new, leading-edge Linux system only to find that it didn't work? Most often this happened during the transition phase from libc4 to libc5 or from libc5 to libc6--the latter of which is also known as glibc (the GNU C library).

The C library is probably the most important library on every UNIX/Linux system because it acts as the interface between each application and the kernel. Changes in such a central library can easily render a system useless if such changes are not backward compatible.

There are other scenarios as well: a program may have been built on a system that was using newer versions of libraries than you have installed on your system. It is likely that such programs will not even start or work properly on your system. This happens if the program is actively using the new or changed features in the updated versions of the libraries.

It is common sense that versions of a library with functionality not included in former versions (and that carry a version number indicating that programs link to an older version) cannot be loaded and executed. This version number is called the major revision of a shared library. The following diagram explains this by using glibc as an example:

GLIBC Version Filename
libc5 /lib/libc.so.5
libc6 /lib/libc.so.6


To maintain certain minor revisions of a shared library, the filename can be augmented by additional suffixes such as libc.so.6.3.3.

As described in the "Background" section, there are additional complications caused by binary incompatibilities of C++ libraries compiled with GNU C++ compilers prior to version 3.3. Even with a currently standardized environment including consistent Application Programming Interface (API) and Application Binary Interface (ABI) definitions, it is not yet clear if future versions of C++ compilers and libraries will remain completely compatible. To address such issues, the Linux Standards Base (LSB) recommends that every software vendor creating software written in C++ statically link the C++ parts. Unfortunately, very few vendors actually have followed this recommendation. To make things worse, a Linux distributor created its own version of a C++ compiler but failed to ensure that its shared libraries carried different version numbers than those used for the official C++ compiler.

In short, if you want to start a program and it fails to do so (and emits error messages similar to the following), your problem can likely be solved by following the suggestions described in "The Solution" section of this white paper.

Error Message When Starting a C Program

./a.out: relocation error: ./a.out: symbol errno, version GLIBC_2.0 not defined in file libc.so.6 with link time reference

If you happen to see the above error message about the symbol errno, your program has been linked with a version of glibc older than 2.3. Newer versions of glibc no longer provide errno as a global variable to allow its thread-safe usage. In this situation, you would have to set up a compatibility environment with an older version of glibc, such as version 2.2.5.

If, however, you don't see any error message at all and the program starts but doesn't work as you expect it to, you might be experiencing the C++ problem described above. The runtime linker, which is loading your program into memory and resolving any dependencies with shared libraries, is able to locate the required shared libraries on your system and to load them into memory. The shared libraries may not, however, fit with the actual program.

Diagnosing Problems

The first thing to do in identifying problem sources is to run the command ldd on your programs. The purpose of this program is to print the shared libraries required by each program or the shared library specified on the command line. Running ldd on /bin/bash will show output similar to the following:

ldd /bin/bash
linux-gate.so.1 => (0xffffe000)
libreadline.so.4 => /lib/libreadline.so.4 (0x40036000)
libhistory.so.4 => /lib/libhistory.so.4 (0x40062000)
libncurses.so.5 => /lib/libncurses.so.5 (0x40069000)
libdl.so.2 => /lib/libdl.so.2 (0x400af000)
libc.so.6 => /lib/tls/libc.so.6 (0x400b2000)
/lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x40000000)

If a shared library cannot be found, an error message will be shown:

libfoo.so.1 => not found

This error message indicates that the program you are trying to execute actually needs the shared library libfoo.so.1 but cannot find it in the predefined locations that shared libraries look for. These predefined locations normally contain the directories /usr/lib and /lib. In addition, a system administrator can add additional directories to a file /etc/ld.so.conf to instruct the runtime linker where to search for shared libraries. As a last resort, an environment variable LD_LIBRARY_PATH can be defined to list additional directories. On Linux, this variable will not be used for programs that run under Ids and that do not belong to the user.

The Solution

After you have identified the source of the problem, you can set up a compatibility environment to provide the required functionality to execute your program properly. We strongly advise installing additional libraries and other files outside of the standard system directories /lib and /usr/lib, as shown in the following example:

Program name to be migrated zoo
Prefix ($prefix) directory location to install the compatibility files /opt/compat-env/zoo
List of required libraries glibc-2.3.2-95.27
libgcc-3.2.3-42 libstdc++-3.2.3-42
Source ($srcdir) directory where required files are located initially /root/zoo-src


Step 1

Create directories /bin and /lib under the prefix directory:

% prefix=/opt/compat-env/zoo
% mkdir -p $prefix/bin $prefix/lib


Step 2

Place copies of the required files into their respective locations:

% srcdir=/root/zoo-src
% cd $prefix/bin
% cp -p $srcdir/zoo .
% cd $prefix/lib
% rpm2cpio $srcdir/glibc-2.3.2-95.27*.rpm | cpio -idvm '*libc*.so*'
% find . -name '*libc*.so*' -exec mv -v '{}' . \;
% rpm2cpio $srcdir/glibc-2.3.2-95.27*.rpm | cpio -idvm '*ld*.so*'
% find . -name '*ld*.so*' -exec mv -v '{}' . \;
% rpm2cpio $srcdir/libgcc-3.2.3-42*.rpm | cpio -idvm '*libgcc_so*.so*'
% find . -name '*libgcc_so*.so*' -exec mv -v '{}' . \;
% rpm2cpio $srcdir/libstdc++-3.2.3-42*.rpm | cpio -idvm '*libstdc*.so*'
% find . -name '*libstdc*.so*' -exec mv -v '{}' . \;
% rm -rf lib usr


Step 3

Create a wrapper script to set up the appropriate environment variables and start the program:

% cd $prefix/bin
% mv zoo zoo.exec
% cat > zoo << EOF
#! /bin/bash
prefix=/opt/compat-env/zoo
LD_LIBRARY_PATH="$prefix/lib${LD_LIBRARY_PATH+:${LD_LIBRARY_PATH}}"
export LD_LIBRARY_PATH
$prefix/lib/ld-linux.so.2 ${0}.exec "$@"
EOF
% chmod 755 zoo


Now the program zoo can be executed, and the versions of glibc, libgcc and libstdc++ will be used as per the requirement.

Additional Problems

After you have set up a compatibility environment as described in the previous section, it is still possible that the program cannot be executed. In this instance, an an error messages such as the following is printed:

symbol __libc_wait, version GLIBC_2.0 not defined in file libc.so.6 with link time reference

Newer versions of the GNU C library assign version numbers not only to the library itself as a whole but also to individual symbols. This usually happens to conform with certain language standards. In the example, the global symbol __libc_wait with version number GLIBC_2.0 is not defined anymore. This could be solved by using an older version of the C library or by using another feature of glibc and its runtime linker. It is possible to preload shared objects to override symbols defined in shared objects that would be loaded in the course of the runtime linker's normal dependency resolution. This can also be used to define symbols that are otherwise not defined, such as the __libc_wait function in the example.

The proper fix for this problem would be to create a small shared object containing the required implementation and to load it in the wrapper script:

% cd $prefix/lib
% cat > libcwait.c << EOF
#include 
#include 
#include 
#include 
pid_t __libc_wait (int *status)
{
  int res;
  asm volatile ("pushl %%ebx\n\t"
                "movl %2, %%ebx\n\t"
                "movl %1, %%eax\n\t"
                "int $0x80\n\t"
                "popl %%ebx"
                : "=a" (res)
                : "i" (__NR_wait4), "0" (WAIT_ANY), "c" (status), "d" (0),
                  "S" (0));
  return res;
}
EOF
% gcc -fpic -O2 -shared libcwait.c -o libcwait.so


Now make the required modifications to the wrapper script:

% cd $prefix/bin
% cat > zoo << EOF
#! /bin/bash
prefix=/opt/compat-env/zoo
LD_LIBRARY_PATH="$prefix/lib${LD_LIBRARY_PATH+:${LD_LIBRARY_PATH}}"
export LD_LIBRARY_PATH
LD_PRELOAD=$prefix/lib/libcwait.so
export LD_PRELOAD
$prefix/lib/ld-linux.so.2 ${0}.exec "$@"
EOF
% chmod 755 zoo


Conclusion

Using the suggestions outlined in this white paper, you can create a compatible environment to execute programs that your vendor has not yet ported to or built on a specific version of a Linux operating system. For example, you can easily set up a compatible environment to run programs from a Red Hat Enterprise Linux* system on a SUSE Linux Enterprise Server and vice-versa.

Additional information can be found in the GNU C library documentation, which should be included with your particular Linux distribution, and at the LSB Web site.


Novell Cool Solutions (corporate web communities) are produced by WebWise Solutions. www.webwiseone.com

© 2014 Novell