Identity Manager eDirectory Driver error messages:
Table of Contents:
Novell Identity Manager supports many different drivers. Each one has its own set of subtleties that are unique to the connected system.
This to me, is one of the more interesting parts of working with Identity Manager. Not only do you need to know and understand eDirectory (the underlying data store for all your objects), Identity Manager itself and its components (things like DirXML Script, XSLT, XPATH, etc) but you also need to learn a fair bit about each connected system.
More amusing is that the level at which you need to learn the connected system is quite deep into the system internals, and interesting, but also mostly useless in the day to day operations of that system. For example, an end user could care less about how certificates in Lotus Notes are used with Domino servers. Well for Identity Manager this is a huge deal, and of critical importance. In the case of the SAP HR driver, you need to know a lot about how iDOC's are written, parsed, and used that most SAP HR users and administrators would not even know existed.
With each connected system that comes into your experience, you have to learn more and more about it. Each system has its own set of specific error messages, some are analogous to others (669 in eDir, LDAP 49, subtype 52e in Active Directory for bad password attempts), while others are unique not only to the specific driver but the specific usage case.
On the one hand, I would love if Novell could document all these possible errors. But in reality, the vast majority being connected system specific, ought to be documented in the general case by the connected system vendor. However, that never really works out all that well.
To try and help Novell along, I have been collecting, annotating, and publishing articles on the various different error codes and cases I have run into in the real world.
For the Active Directory driver you can read:
For the JDBC driver you can read:
I have a bunch more collected that I am working on adding comments and explanations to for the eDirectory driver (this article), more Active Directory errors, more JDBC errors, SAP HR driver errors, Group Wise driver errors, and more. Now I just need to find the time to finish writing them all up and submitting them.
You can see my personal collection of articles at: http://wiki.novell.com/index.php/Geoffrey_Carman's_personal_collection
They are sorted by topic, and the name of the article usually indicates the topic. I find this view easier than my author page at Cool Solutions: http://www.novell.com/communities/user/555/track
I highly recommend that if you happen to working with a new driver, that you keep a good text editor open (I use TextPag, but others like NotePad or even Gedit in SLED) and when you see an error in Dstrace, copy and paste it into the text file. Once you figure out what happened, write down enough details to remind yourself of the cause, and then the resolution so that you can share it like this with other people. This way, everyone benefits as the next time they do a Google search for the error string they can find an article that talks about the error.
If you are not aware of how to troubleshoot Identity Manager drivers, then I highly recommend you read the following set of articles. First David Gersic's truly excellent walk through series of what all the 'things' or steps in the Identity Manager process (as visualized by the fishbone diagram in iManager or Designer):
That will get you up to speed on what is supposed to be happening under the covers. Then you need to look at what actually happened under the covers by watching the event happen in Dstrace. To get up to speed on reading Dstrace, the best article I have ever read on the topic is by Fernando Frietas, a support engineer in Novell Technical Services: Capturing and Reading Novell Identity Manager Traces
Forget about the logs, you need to read the Dstrace output. To see all the ways you can read Dstrace, you could try this shorter article: The Many Faces of DSTRACE
Finally, if you are having issues, have looked at the trace, and cannot see a problem, or you have searched and read about your specific issue and it is still not working for you, then you should consider posting in the Novell Support Forums at http://forums.novell.com or via a news reader at nntp://forums.novell.com and there is a really nice and active forum that many people read and post too, novell.support.identity-manager.engine-drivers
Now with that basic introduction complete, lets get into some error codes and what is going with them:
DirXML Log Event -------------------
Driver: \WATTS-LAB-IDV\Watts\Drivers\IDM\eDirectory
Channel: Subscriber
Status: Error
Message: Code(-9075) Shutting down because DirXML engine evaluation period has expired. Activation is required for further use.
This is a pretty generic error, and worth mentioning in an eDirectory specific error list, since it is actually the engine, not the driver per se that has expired.
When you install Identity Manager, you get a 90 day license for free, after which you have to activate it. You use iManager, and when you look at the Identity Manager Overview, and select a driver set, you should see a warning that the engine or driver is not activated yet. You then go to Novell's Customer Center to get the base 64 encoded activation credential to paste into the iManager Activation option. In this case, it was our lab, and we forgot to actually activate it, before time ran out on us. Oops.
DirXML Log Event -------------------
Status: Error
Message: Code(-9067) Error while initializing drivers: VR Driver Interface Module not loaded (-783)
15:12:30 9E62D5E0 Drvrs: Error initializing DirXML: com.novell.nds.dhutil.DSErr: VR Driver Interface Module not loaded (-783)
at com.novell.nds.dirxml.engine.MiscDS.translateException(MiscDS.java:472)
at com.novell.nds.dirxml.engine.MiscDS.setDriverState(MiscDS.java:196)
at com.novell.nds.dirxml.engine.DirXML.initializeDrivers(DirXML.java:673)
at com.novell.nds.dirxml.engine.DirXML.access$500(DirXML.java:42)
at com.novell.nds.dirxml.engine.DirXML$DriverStarter.run(DirXML.java:924)
at java.lang.Thread.run(Thread.java:534)
Caused by: novell.jclient.JCException: request -783 ERR_VRDIM_NOT_INITIALIZED
at novell.jclient.JClient.request(Native Method)
at novell.jclient.JClient.ndsRequest(JClient.java:1197)
at com.novell.nds.dirxml.engine.MiscDS.setDriverState(MiscDS.java:181)
... 4 more
The 783 error code has a number of possible causes. VRDIM could really not be loaded. Then the trick is to figure out why. Maybe someone unloaded it? Maybe you installed the wrong version? We ran into that last case where a patch had an eDir 8.7.3 version and an eDir 8.8 version of the patch, and we installed the wrong RPM on SLES and started getting crazy errors like this.
Once you assign a driver set to a server, it will then start auto loading vrdim, and dxevent as soon as eDirectory starts on that server. (On Netware it is an NLM, a DLM on Windows, and an .so file on the Unix flavours).
15:23:25 8D0093E0 DirXML:
DirXML Log Event -------------------
Driver: \ACME-META\CIM\DirXML\Active Directory (acme.corp)
Status: Error
Message: (-9947) Client request for invalid state transition from 0 to 3.
15:23:31 84793400 DirXML:
DirXML Log Event -------------------
Driver: \ACME-META\CIM\DirXML\Active Directory (acme.corp)
Status: Error
Message: (-9947) Client request for invalid state transition from 1 to 1.
However, in this case, we had a different problem, that manifested itself with the initial 783 and then further errors like this, as we tried to start drivers.
DirXML Log Event -------------------
Status: Error
Message: Code(-9140) Error processing DirXML sub-verb DSVR_GET_DRIVER_STATS: com.novell.nds.dhutil.DSErr: no such entry (-601)
at com.novell.nds.dirxml.engine.cache.DriverCache.getTransactionStats(Native Method)
at com.novell.nds.dirxml.engine.verb.GetDriverStats.version0(GetDriverStats.java:249)
at com.novell.nds.dirxml.engine.verb.GetDriverStats.processSubVerb(GetDriverStats.java:178)
at com.novell.nds.dirxml.engine.verb.DirXMLVerbs$GetVerbHandler.processVerb(DirXMLVerbs.java:530)
at com.novell.nds.dhutil.VerbProcessor$HandlerThread.run(VerbProcessor.java:507)
at java.lang.Thread.run(Thread.java:534)
Module DXEVENT.NLM load status OK
Loading module DXLDAP.NLM
DirXML Event Handler for Novell Directory Services 3.5.1
Version 3.05.10 September 18, 2007
Copyright 1999-2007 Novell, Inc. All rights reserved. Patents Pending.
SERVER-5.70-918: Loader cannot find public symbol: NLDAPSetResponseBer for module DXLDAP.NLM
SERVER-5.70-918: Loader cannot find public symbol: NLDAPGetBerFromHandle for module DXLDAP.NLM
SERVER-5.70-918: Loader cannot find public symbol: NLDAPSendResult for module DXLDAP.NLM
SERVER-5.70-918: Loader cannot find public symbol: NLDAPGetContext for module DXLDAP.NLM
SERVER-5.70-918: Loader cannot find public symbol: NLDAPBerAlloc for module DXLDAP.NLM
SERVER-5.70-918: Loader cannot find public symbol: NLDAPBerFree for module DXLDAP.NLM
SERVER-5.70-918: Loader cannot find public symbol: NLDAPBerPrintf for module DXLDAP.NLM
SERVER-5.70-918: Loader cannot find public symbol: NLDAPBerScanf for module DXLDAP.NLM
SERVER-5.70-918: Loader cannot find public symbol: NLDAPFree for module DXLDAP.NLM
SERVER-5.70-918: Loader cannot find public symbol: NLDAPIsSecureConnection for module DXLDAP.NLM
Error processing External Records.
Module DXLDAP.NLM NOT loaded
Module DXLDAP.NLM load status UNRESOLVED
Here we saw that DXEVENT.NLM file on Netware would not load as it could not find some symbols it needed from the DXLDAP.NLM
But the actual root cause was one step further down, which we found as DIRXML.NLM (VRDIM for Netware) loaded.
Loading module DIRXML.NLM
Novell Identity Manager 3.5.1
Version 3.05.10 September 18, 2007
Copyright 1999-2007 Novell, Inc. All rights reserved. Patents Pending.
Auto-Loading Module JCLNT.NLM
Auto-loading module JCLNT.NLM
NetWare JClient-Native (Build 1.5.1279)
Version 1.05 September 19, 2007
Copyright 1999 Novell, Inc. All rights reserved.
Auto-Loading Module JCLNTR.NLM
Auto-loading module JCLNTR.NLM
NetWare JClient-Native Resources (Build 1.5.1279)
Version 1.05 September 19, 2007
Copyright 1999 Novell, Inc. All rights reserved.
Module JCLNTR.NLM load status OK
Module JCLNT.NLM load status OK
Module DIRXML.NLM load status OK
Novell Audit Platform Agent: Failing primary connection for application DirXML.
Loading module DHUTILJ.NLM
The Audit connection had a problem. When you install Identity Manager you can choose to install the Audit components or not. They are useful if you are collecting the events to something. Used to be Audit 2.0 but that has since been replaced with Sentinel 6.x or Identity Audit. Both can collect events from the Novell Audit collectors.
14:05:56 B8824140 DirXML:
DirXML Log Event -------------------
Status: Error
Message: (-9983) An error occurred while logging to Novell Audit: failed, 11 (0xb).
DSTrace showed an error connecting to Novell Audit.
Looking back at the server console for errors, and we saw:
Novell Audit Platform Agent: All log channels have failed. Stopping logging of events for application DirXML.
Novell Audit Platform Agent: All log channels have failed. Stopping logging of events for application DirXML.
Novell Audit Platform Agent: ACK Failure for Driver\%s\Subscriber
Loading module LCACHE.NLM
Nsure Audit Platform Cache Module (Build 55)
Version 2.00.02 September 26, 2008
(c)2003-2006 Novell, Inc. All Rights Reserved.
Module LCACHE.NLM load status OK
Novell Audit Cache: Log Cache Dir : sys:/etc/logcache
Novell Audit Platform Agent: All log channels have failed. Stopping logging of events for application DirXML.
Novell Audit Platform Agent: All log channels have failed. Stopping logging of events for application DirXML.
Novell Audit Platform Agent: ACK Failure for Driver\%s\Subscriber
SERVER-5.70-151: Unable to find load file SYS:/SYSTEM/LCACHE.NLM
Novell Audit Platform Agent: Failed to connect to cache for application DirXML,
DISABLING cache mode.
Novell Audit Platform Agent: All log channels have failed. Stopping logging of events for application DirXML.
Novell Audit Platform Agent: All log channels have failed. Stopping logging of events for application DirXML.
Loading module AUDITEXT.NLM
Novell Nsure Audit Schema Tool
Version 2.00.02 September 26, 2008
(c)2003-2006 Novell, Inc. All Rights Reserved.
Auto-Loading Module NWSNUT.NLM
Auto-loading module NWSNUT.NLM
NetWare NLM Utility User Interface
Version 7.00.01 October 26, 2005
Copyright 1989-2005 Novell, Inc. All rights reserved.
Module NWSNUT.NLM load status OK
Auto-Loading Module MDB.NLM
Auto-loading module MDB.NLM
Multiple Directory Database (Build )
Version 2.00.02 June 28, 2006
(c)2003-2006 Novell, Inc. All Rights Reserved.
Module MDB.NLM load status OK
Loading module MDBDS.NLM
Module AUDITEXT.NLM load status OK
MDB eDirectory Driver (Build )
Version 2.00.02 June 28, 2006
(c)2003-2006 Novell, Inc. All Rights Reserved.
Module MDBDS.NLM load status OK
MDBDriver 'mdbds.nlm
Here we can see all sorts of issues with connecting to the Audit server.
14:32:16 85029380 Drvrs: DirXML starting.
14:32:16 85029380 Drvrs: Unable to load Novell Audit LogEvent module: failed, -5984 (0xffffe8a0)
14:32:26 867841E0 Drvrs: DirXML engine thread starting.
Finally we renamed logevent.nlm and we could start the engine up. Since this was a lab anyway, we did not care, as we did not actually have anywhere for the Audit components to log too. We never did track down what the actual cause was, but the progression of trace and logs above is nice, as it shows each step of the failure, and how a high level 783 error bubbled up from a much more low level problem of logevent.nlm being unable to connect to a secure logging server.
[05/05/09 17:19:54.742]:eDirectory ST:
<nds dtdversion="3.5" ndsversion="8.x">
<source>
<product version="3.6.1.4427">DirXML</product>
<contact>Novell, Inc.</contact>
</source>
<input>
<move class-name="User" dest-dn="\ACME-LAB-LDAP\corp\acme\asiapac\Japan\tgwuser3" dest-entry-id="33907" event-id="m
ta-gwlab#20090505211924#1#2">
<association>{AAF66FC2-236A-2c4d-949C-AAF66FC2236A}</association>
<parent dest-dn="\ACME-LAB-LDAP\corp\acme\asiapac\Japan\tgwuser3"/>
</move>
</input>
</nds>
[05/05/09 17:19:54.744]:eDirectory ST: Pumping XDS to eDirectory.
[05/05/09 17:19:54.744]:eDirectory ST: Performing operation move for \ACME-LAB-LDAP\corp\acme\asiapac\Japan\tgwuser3.
[05/05/09 17:19:54.745]:eDirectory ST: Moving entry \ACME-LAB-LDAP\corp\acme\asiapac\Japan\tgwuser3 to \ACME-LAB-LDAP
\corp\acme\asiapac\Japan\tgwuser3.
[05/05/09 17:19:54.753]:eDirectory ST: Processing returned document.
[05/05/09 17:19:54.753]:eDirectory ST: Processing operation <status> for .
[05/05/09 17:19:54.753]:eDirectory ST:
DirXML Log Event -------------------
Driver: \ACME-LAB-LDAP\acme\Drivers\IDM\eDirectory
Channel: Subscriber
Status: Error
Message: Code(-9010) An exception occurred: novell.jclient.JCException: moveEntry -611 ERR_ILLEGAL_CONTAINMENT
In this case I was trying to automate a move based on some attribute changing, like L (Location) changing, means move the user to a different container. But in this trace sample it is trying to move a User object into a User object. That won't work! Its illegal for a User to contain another user. I.e. Users are not usually containers (except in dumbo implementations like one of the PBX/VOIP phone vendors did, where they made users that contained objects with settings, and I think I recall seeing a forum post where Citrix might try adding objects to store configuration settings on users as well). What I should have done was used ParseDN to chop off the user CN from the target DN, (i.e. Specify the container I wanted to move the user into, not the destination complete path). In this case, since the DN is the same for the source user and the destination path, it would actually return a -606 Object already exists if I had tried.
I really did need to fix that rule, this was just my first draft, not really thought through try at it.
[05/07/09 12:51:36.748]:eDirectory ST:
<nds dtdversion="3.5" ndsversion="8.x">
<source>
<product version="3.6.1.4427">DirXML</product>
<contact>Novell, Inc.</contact>
</source>
<input>
<modify class-name="User" dest-dn="\ACME-LAB-LDAP\corp\acme\americas\West\AMES\Users\tatert" dest-entry-id="33780" event-id="mta-gwlab#20090507165136#1#3">
<association>{38D65D96-C456-824d-3988-38D65D96C456}</association>
<modify-attr attr-name="acmeCrossDomainMoveReset">
<remove-all-values/>
<add-value>
<value>\ACME-LAB-LDAP\corp\acme\emea\France\Fried\tatert</value>
</add-value>
</modify-attr>
</modify>
</input>
</nds>
[05/07/09 12:51:36.765]:eDirectory ST: Pumping XDS to eDirectory.
[05/07/09 12:51:36.765]:eDirectory ST: Performing operation modify for \ACME-LAB-LDAP\corp\acme\americas\West\AMES\Users\tatert.
[05/07/09 12:51:36.767]:eDirectory ST: Modifying entry \ACME-LAB-LDAP\corp\acme\americas\West\AMES\Users\tatert.
[05/07/09 12:51:36.806]:eDirectory ST: Processing returned document.
[05/07/09 12:51:36.806]:eDirectory ST: Processing operation <status> for .
[05/07/09 12:51:36.806]:eDirectory ST:
DirXML Log Event -------------------
Driver: \ACME-LAB-LDAP\acme\Drivers\IDM\eDirectory
Channel: Subscriber
Status: Error
Message: Code(-9010) An exception occurred: novell.jclient.JCException: modifyEntry -608 ERR_ILLEGAL_ATTRIBUTE
[05/07/09 12:51:36.838]:eDirectory ST: Direct command from policy result
[05/07/09 12:51:36.838]:eDirectory ST:
<nds dtdversion="3.5" ndsversion="8.x">
<source>
<product version="3.6.1.4427">DirXML</product>
<contact>Novell, Inc.</contact>
</source>
<output>
<status event-id="mta-gwlab#20090507165136#1#3" level="error">Code(-9010) An exception occurred: novell.jclient.JCException: modifyEntry -608 ERR_ILLEGAL_ATTRIBUTE<application>DirXML</application>
<module>eDirectory</module>
<object-dn></object-dn>
<component>Subscriber</component>
</status>
</output>
</nds>
In this case, I was trying to manage three connected Active Directory domains, syncing into a single flat eDirectory based Identity Vault. Moves within domains were mapped into changes of the DirXML-ADContext attribute, and then when it hit the eDirectory to eDirectory driver that change of the attribute DirXML-ADContext became a move event.
However, moves between Active Directory domains was a concern, that would be hard to handle, so I added a flag attribute, when I detected that error case of acmeCrossDomainMoveReset to store where the user was, when it suddenly appears in the new domain.
However, I had created the attribute as part of an auxiliary class, that the user did not yet have. Usually the engine will try and add the attribute, but in this case it had not. Seems like if you add an attribute on a destination eDirectory, then the engine will add the needed auxiliary classes as appropriate. However when writing back to the source eDirectory, looks like you need to manage that yourself and add it to the object. Once that was done, this worked.
Without the auxiliary class, the attribute is Illegal, since it is not part of any classes for the target object.
[05/07/09 13:08:08.106]:Generic Null ST:Driver object has insufficient rights to read \ACME-LDAP\corp\acme\emea\France\Fried\tatert#acmeCrossDomainMoveReset.
I forget this one so often! When you import or deploy a new driver, you have to set security equals for the driver object, so that it has sufficient rights to operate (Read and Write) within the directory. If you do not do that, you often see nothing, since it does not have rights to even see the event, or else when you try to write out a value, you get an error as above.
Nice clear error. Unfortunately being a single line long, it often gets buried in mountains of trace, and can be hard to find or track down.
You can use iManager or Designer to add Security equivalence for the driver, or you can just set the Security Equals attribute on the driver object directly via ConsoleOne or via an LDAP tool.
In this particular case, what happened was in the lab, to be lazy we set Security Equals on the driver set, and thus all the drivers inherited that right from their parent object, the driver set. But when we deployed to production, we forgot to set the equivalence on the driver as was needed.
<nds dtdversion="3.5" ndsversion="8.x">
<source>
<product version="3.6.1.4427">DirXML</product>
<contact>Novell, Inc.</contact>
</source>
<output>
<status event-id="AMERICAS-AD##121360cbf4d##1" level="error">Code(-9172) Error in CN=CN=Query Smithers,OU=Users,OU=SMITH,OU=East,DC=americas,DC=acme,DC=corp : An invalid DN '{1}' is specified: DN does not conform to the format required by the current context.<application>DirXML</application>
<module>eDirectory</module>
<object-dn>\ACME-LAB-IDV\Watts\Users\qsmith (corp\acme\americas\East\SMITH\Users\qsmith)</object-dn>
<component>Publisher</component>
</status>
</output>
</nds>
This was triggered by a change in the DirXML-ADContext on the eDirectory side, sending into an Active Directory driver (which is why I had included it in this article, though really it is an Active Directory driver error, but lets not get picky, ok?)
If you look at the DN in the error message, you will see it is CN=CN= which is clearly wrong. When using ParseDN, there are a number of options. You can read more in these articles:
In this case, when you convert from source to destination format (the default if you change nothing) if the source is eDirectory, then there will not be a fully qualified name version (the CN=This,OU=that,o=there and so on, rather it will be \there\that\This) which also applies if you are using ParseDN start of -1 and length of 1 to get the objects name. In the LDAP case, you would get CN=This whereas in an eDirectory case you would end up with just This. So depending on your source and destination, remember to set the conversion types correctly. In this case, I had rebuilt the DN by hand in Policy, so I had pre-prended a CN= before the local variable where I had the object name resulting from the ParseDN token. Thus I ended up with an extra CN=.
Thats it for now, stay tuned for part 2 of this article where I work through even more errors I ran into in the real world, using the eDirectory driver for Identity Manager.
Support
Documentation
Learning Services
Partner Programs