"Error: File Protocol Error Occurred ..." messages when using Storage, Archive Versioning, File Protocols, and Clustering Plug-Ins for iManager.

  • 3417215
  • 07-Mar-2008
  • 04-Feb-2013

Environment

Novell Open Enterprise Server (OES)
Novell Linux

Situation

"Error: File Protocol Error Occurred ..." messages when using Storage, Archive Versioning, File Protocols, and Clustering Plug-Ins for iManager.
or
"This user does not have the correct credentials to authenticate to the cimom client"

Resolution

Basic Information:

This is a generic error message. This error means that we could not communicate with the server using any of the following protocols, listed below in the order they are tried.

  1. CIMOM (OpenWBEM Common Information Model Object Manager)

  2. NCP (Novell Core Protocol)

  3. CIFS (Common Internet File System)

In iManager there are three plugins that use this same authentication mechanism.

  1. Archive Versioning (Archive Versioning plugin)

  2. Clusters (Cluster Services Management plugin)

  3. Storage (Storage Management plugin)

IMPORTANT:The storage-related plug-ins share common code in the storagemgmt.npm file. If you use more than one of these plug-ins, you should install, update, or remove them all at the same time to make sure the common code works for all plug-ins.

NOTE: You need to add to reference the new plugin (Common Storage plugin, storagemgmt.npm).

If the common plugin is not installed, the other three plugins will not work.

The common code is only deleted if the storagemgmt.npm is uninstalled.

If you update the storagemgmt.npm, you may cause any of the other three not to work if they are not updated.



Troubleshooting:
Follow link for NetWare troubleshooting NetWare 6.5 Troubleshooting
  • Make sure they are on iManager 2.5 that ships with OES SP2 or later and the plugins need to end with 20051213 (or later) as see in in iManager | Configure | Module Installation | Installed Novell Plug-in Modulesā€. Look at Archive Versioning, Cluster Services, and Storage Management.
  • Only test with the admin user or admin equivalent.
  • Confirm that the IP address that is assigned to the Master IP Address Resource is bound to the server that is running the Master IP Address Resource. Use "ip addr show | grep secondary" and you should see it bound.  The Master IP address needs to be the same in the cluster load and unload script, in the attributes of the cluster object in "NCS:Network Address" and "Network Address" with UDP and TCP.
  • Make sure user is LUM enabled

    • use "id USERā€ where USER is the user they are using to authenticate. Compare this UID and GID with the LUM information in eDirectory. If this step does not work, check to see if namcd is running.
      • "/etc/init.d/namcd status"
      • If it is not running, start it with "/etc/init.d/namcd start"
      • If it is dead, restart it with "/etc/init.d/namcd restart"
      • In some cases it may be nessasary to point lum to a diffrent ldap server.
    • In iManager go to eDirectory Administration | Modify object | Select your USER. In the pull down menu select "Linux Profile Pageā€ compare this UID and GID to the "idā€ output above. If they are the same then LUM is working if not then fix LUM for this user.

    • verify that the user id is unique (admin.novell and admin.users.novell cannot both be lum enabled - that would create more than one 'admin' user in linux with two different uid's.)

  • Check to make sure OpenWBEM is a LUM enabled service.   Default OES installation the service is enabled.   You may check the /etc/sysconfig/novell/lum1 file for the following.
    CONFIG_LUM_SERVICE_OPENWBEM="yes"
    You can also check the Unix Workstation object in iManager.  Using iManager select  "Linux User Management" and Modify Unix Workstation Object.  Select the Unix Workstation object for the server.  In the Linux Profile you should see which LUM Enabled Services are enabled.  Make sure "openwbem" shows up in this list.  Also make sure "admingroup" shows up in the Groups section as well.   By default, this service is enabled when installing OES.
    If it's not configured you should run 'YAST2 novell-lum' and LUM-enable the OpenWBEM service.

    With debug options enabled for openwbem you will see in the output log that the user root will fail to authenticate where the user logged into iManager is the user it should be using.   
    Example output from /var/log/messages with debug enabled for openwbem.
    openwbem: [1333913920]HTTPServer::authenticate: processing OWLocal
    openwbem: [1333913920]HTTPServer::authenticate: authentication failed for: root

  • Check the "Network" attributes on the cluster container object.  First check "NCS:Network Address" attribute and make sure the hex translates into the IP Address for the Master IP Resource.  Next, check the "Network Address" attribute which has the TCP and UDP port 524 addresses for the Master IP Resource address as well.  These addresses should match the "NCS:Network" address.  If these addresses differ, manually change them to match.   This could be the case if the IP address was ever changed on the cluster.  
    Symptom:  iManager Storage plugin is working but the iManager Cluster Options or Cluster Manager does NOT work.  When you select the cluster object,  the screen appears to hang and takes 2-3 minutes to time out.  It doesn't show the typical file protocol error or any other error very useful.   

  • Check the /var/log/messages file for "ldap_initconn" you may find several lines similar to the following:
    Mar 17 14:18:16 SERVER1 /usr/sbin/namcd[4965]: ldap_initconn: LDAP bind failed (error = [81]), trying to connect to alternative LDAP server"
    If so then there is a LUM problem.  Follow the information in
      TID 3401691 - namcd cannot connect to LDAP server

    This error can also occur (with LUM working properly) if the server is trying to connect to eDirectory/ldap over port 636, but is unable to do so.  Check the /etc/nam.conf for the "type-of-authentication=" and make sure this is set to 2 (and not 1).  Option 1 is for port 389 and option 2 is for port 636 (or secure ldap).  A workaround, or another possible solution for this, is to make sure that "Require TLS for Simple Binds with Password" is not checked under the ldap group object for the server you are trying to authenticate into.
  • Test each of the three plugins (Archive Versioning, Clusters, Storage) to see if you get the "File Protocol Error ...ā€ If you get the error on only one of the plugins then you know that it is specific to that plugin, and the authentication protocols are fine (CIMOM, CIFS, NCP). Then troubleshoot using the following steps
  • Clusters
    Check /etc/opt/novell/ncs/clstrlib.conf file on all cluster nodes. If changes are made then the server needs to be rebooted for the changes to take effect.

(iclstrlibss

data

p0

(dp1

S'adminDn'

p2
S'YmFzZTY0Y249YWRtaW4sbz1ub3ZlbGw=\n' This is the user given to clustering duringinstallation usually the person who installed clustering, ie admin.It is encrypted with post SP2 updates.

p3

sS'ldapUrl'

p4

S'ldaps://151.155.247.116:636'This IP address needs to point to a Master or R/W replica that can respond via LDAP, by default it chooses to its own IP address.

p5
sS'adminPw'

p6

S'YmFzZTY0bm92ZWxs\n'This is the encrypted password of the person that installed clustering. If this password has changed then put it inside the single quotes in clear text and it will become encrypted after reboot.

p7

sS'clusterDn'

p8

S'cn=wss-linux-cluster,o=novell'Confirm that the cluster object has not been moved and this is correct.

p9

sb.

  • Storage and Clustering

    • Turn on VFS (Virtual File System) logging in NSS. Use 'screen' to gather the output of a terminal session.

      • On the server that you browse to in iManager (when using the storage plugin). Start the 'screen' program with "screen -Lā€ (the prompt will move to the top of the screen. This means the 'screen' session is active)

      • Start "nssconā€

      • Enable vfs logging with "/vfsā€ (this will just give you a blank prompt). Leave this terminal open. (The vfs messages are also sent to /var/log/messages)

      • Reproduce the "File Protocol Errorā€ using the storage plugin. You should see XML scrolling in the nsscon window.
        • If you see XML comming out when you duplicate the file protocol error then the problem is with the application (clustering, NSS, etc.).
        • If you do not see XML then it is not getting through the communication protocol (which is usually going through the CIMOM, so troubleshoot the CIMOM piece)
      • Turn off vfs logging with"/novfsā€

      • exit NSSCON with "exitā€

      • exit screen logging with "exitā€

      • get the /root/screenlog.0 file that holds the debug information.


  • Use the following steps, only if all three plugins show the error. We will handle each protocol individually. This should be done on the server that is holding the"Master_IP_Address_Resourceā€ as seen by the "cluster resourcesā€ command (all iManager clustering is done via the server holding the Master_IP_Address_Resource). You will only see a Master_IP_Address_Resource on a system that is running clustering.

    • CIMOM

      • Check to see that CIMOM is running with "rcowcimomd statusā€ if it is then restart it with "rcowcimomd restartā€ if it is not running the start it with "rcowcimomd startā€

      • Test again in iManager if it fails then we need to put both iManager and CIMOM in debug mode and reproduce the error and send engineering the log files.

        • Place iManager into debug mode.

          • Click the Configure icon in iManager (Looks like a person at a desk)

          • Select iManager Server | Configure iManager

          • Choose the Logging Events tab

          • Change the logging level to "Errors, Warnings and Debug Information messages"

          • Click "Saveā€

          • Under Logging Output choose clear.

        • Place owcimomd in debug mode.

          • "rcowcimomd stopā€

          • Then start the process manually, placing it into debug mode and piping the log to a file with "/usr/sbin/owcimomd -d > /tmp/owcimomd.log 2>&1ā€ you will be moved to the next line but not get a prompt, just leave this terminal open.

        • Reproduce the "File Protocol Errorā€.

        • CTRL-C the owcimomd process

        • Have the customer send us /tmp/owcimomd.log and /var/opt/novell/iManager/nps/WEB-INF/logs/debug.html (from the server running iManager as seen in the URL)

        • Restart CIMOM with "rcowcimomd startā€

        • Go back into iManager and place the Logging Level back to "Error", and save it.

    • NCP

      • NCP starts with edirectory. Make sure it is running with "rcndsd statusā€

      • Ensure that ncp2nss is running with "ps aux | grep ncp2nssā€

    • CIFS (NetWare only)

Investigate CIMOM (/tmp/owcimomd.log) debug file.
  • Check to see that the CIMOM is receiving connections from iManager.
    You'll see something like this:

[47430440190288] Received connection on 151.155.188.5:5989 from 137.65.59.11:58376

  • If you don't see this, iManager is not connecting to the CIMOM. Check IP addresses, firewalls, other usual suspects. If you are receiving connections, look for authentication failures, or other instances of the word "fail" or "Fail" that may be related to the NSS admin volume provider. "
  • If you are on a 64 bit server then you could possibly not have the owcimomd providers that you need. 
    • The fix is to creae a symbolic link "ln -s /opt/novell/lib64/openwbem/c++providers/libNovell_NSS_AdminSession.so
      /opt/novell/lib/openwbem/c++providers/"
    • If you see the following in the owcimod debug log then you have this problem.
      [1149274432] XMLExecute::processSimpleReq caught CIM exception:

      Code: 11

      File: OW_InstanceRepository.cpp

      Line: 328

Investigate iManger (/var/opt/novell/iManager/nps/WEB-INF/logs/debug.html) debug file.
  • Search for "CIMOMā€ or "NSSā€. Then look for specific errors when it tries to login.

Trying CIM/XML protocol
11/10/06 [11:26:37.077] VirtualFile...........-1 Exception caught trying CIMOM protocol: 30602
11/10/06 [11:26:37.078] VirtualFile...........-1 Trying NCP protocol
11/10/06 [11:26:37.080] VirtualFile...........-1 Exception caught trying NCP protocol: 30602

The 30602 is an access denied exception that the iManager plugin throws

Below is a list of error messages and their meanings

ERR_NAMESPACES_NOT_SETUP= 30500,

ERR_CANNOT_OPEN_FILE= 30501,

ERR_CANNOT_OPEN_NSS_MANAGE= 30502,

ERR_CANNOT_OPEN_NDS_MANAGE= 30503,

ERR_BAD_FILE_TO_USE= 30504,

ERR_BAD_CONTEXT= 30505,

ERR_OPEN_FILE= 30506,

ERR_CLOSE_FILE= 30507,

ERR_READ_FILE= 30508,

ERR_WRITE_FILE= 30509,

ERR_MANAGEMENT_PATH_NOT_FOUND= 30510,

ERR_CANNOT_ACCESS_MANAGEMENT_FILE= 30511,

ERR_CANNOT_CLOSE_MANAGEMENT_FILE= 30512,

ERR_CANNOT_ACCESS_RMI_SERVER= 30513,

ERR_CANNOT_GET_TREE_NAME= 30514,

ERR_SESSION_LOGOUT_FAILED= 30515,

ERR_CANNOT_OPEN_NSS_VERSION_FILE= 30516,

ERR_INVALID_NSS_VERSION= 30517,

ERR_FILE_SEEK= 30518,

ERR_FILE_GET_POSITION= 30519,

ERR_INVALID_MANAGEMENT_FILE= 30520,

ERR_BAD_SERVER_NAME= 30521,

ERR_MISSING_OS_TYPE= 30522,

ERR_UNSUPPORTED_OS_TYPE= 30523,

ERR_FILE_ALREADY_OPEN= 30524,

ERR_UNSUPPORTED_ENCODING= 30525,

ERR_BAD_VOLUME_NAME= 30526,

ERR_CANNOT_OPEN_NSS_USER= 30527,

ERR_MISSING_SERVER_NAME= 30528,

ERR_CANNOT_OPEN_NSS_FILES= 30529,


/* ARK File errors */

ERR_CANNOT_OPEN_ARK_MANAGE= 30530,


/* NCS and BCC File errors */

ERR_CANNOT_OPEN_NCS_VERSION_FILE= 30550,

ERR_INVALID_NCS_VERSION= 30551,

ERR_CANNOT_OPEN_NCS_CONFIG_FILE= 30552,

ERR_CANNOT_OPEN_NCS_MANAGE_FILE= 30553,

ERR_CANNOT_OPEN_NCS_REPORT_FILE= 30554,

ERR_CANNOT_OPEN_NCS_MANAGE= 30555,

ERR_CANNOT_OPEN_BCC_VERSION_FILE= 30556,

ERR_INVALID_BCC_VERSION= 30557,

ERR_CANNOT_OPEN_BCC_CONFIG_FILE= 30558,

ERR_CANNOT_OPEN_BCC_MANAGE_FILE= 30559,


/* authentication errors */

ERR_INVALID_USER= 30600,

ERR_INVALID_CREDENTIALS= 30601,

ERR_ACCESS_DENIED= 30602,

ERR_INVALID_CIMOM_ERROR= 30603,

ERR_INVALID_READ_RETURN_PKG_ERROR= 30604,

ERR_UNABLE_TO_SETUP_AUTHENTICATION= 30605,


/* iManager errors */

ERR_MISSING_AUTH_BROKER= 30650,

ERR_MISSING_PLUGIN_BROKER= 30651,

ERR_INVALID_NAMESPACE_OBJECT= 30652,

ERR_INVALID_PLUGIN_NAMESPACE= 30653,

ERR_INVALID_SERVER_OBJECT= 30654,

ERR_INVALID_OBJECT_TYPE= 30655,

ERR_MISSING_IPADDRESS= 30656,

ERR_INVALID_USER_OBJECT= 30657,

ERR_INVALID_TREE_OBJECT= 30658,

ERR_INVALID_VOLUME_OBJECT= 30659;

NOTE: When we attempt to talk to the admin volume from iManager, we get the credentials (username, password), IP address from iManager. So you can also encounter errors when trying to get this information from iManager (30650 ā€“ 30659). If we successfully get this information from iManager, then we try to authenticate to the admin volume using the protocols (CIMOM, NCP and CIFS). CIMOM exceptions are 30600 ā€“ 30605.

All protocols can throw the read, write, open, position, seek and close errors.

The unsupported OS type is generally used when we have something that does not run on Linux and NetWare.

  1. Confirm that the cimom is running with "modules owcimomd". It should come bac with a few lines.
  2. If the owcimomd is not running type "openwbem.ncf" and then recheck if owcimomd is running.
  3. Determine if the same user is starting the comom? Check the sys:\system\cimom\etc\openwbem\openwbem.conf file.
    1. owcimomd.allowed_users = * or what ever user has rights to the cimom.
    2. http_server.http_port = -1
    3. http_server.https_port = 5989. It might have a ; at the start of the line meaning that is commented out.
    4. http_server.max_connections = 30. It might have a ; at the start of the line meaning that is commented out.
    5. Try changing the following variable in the openwbem.conf file "owcimomd.allow_anonymous = true" and restart openwbem and test again.  This allows a client to bypass secure authentication.   If this works and /or you've changed or updated your server certificates using pkidiag recently, follow the directions in step 6.
    6. Use the bash console in NetWare to recreate the *.pem certificate file that openwbem uses.  Type "bash" at the console to open a new bash shell.  This should open a screen and be at a "#" prompt in the current working directory of  "/usr/home".  Change into the following directory. "cd /system/cimom/etc/openwbem/ "   In this directory is a bash script which can be used to recreate the "hostkey+cert.pem" certificate file.  From this directory inside the bash shell type "./owgencert --force".  You can do a "dir" listing which should show a new file date on the "hostkey+cert.pem" certificate file.
If the above steps do not fix the problem, please send Novell technical support the syslog (by default located at sys:\system\cimom\var\owcimomd.log) The owcimomd.conf file will tell what type of logging the cimom is using.  Please enable the DEBUG logging before sending in the owcimomd.log file.  This can be done with the following variable in openwbem.conf file.
log.main.level = DEBUG

Status

Top Issue

Additional Information

Formerly known as TID# 10100891